[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:28:36 dist-test-slave-w206 krb5kdc[7676](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:28:36 dist-test-slave-w206 krb5kdc[7676](info): set up 2 sockets
Aug 08 14:28:36 dist-test-slave-w206 krb5kdc[7676](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 14:28:38.025060 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.007s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:28:38 dist-test-slave-w206 krb5kdc[7676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:28:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:28:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
W20260808 14:28:38.402141 7669 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.7.125.126: real 0.268s user 0.004s sys 0.000s
I20260808 14:28:38.403767 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:32789
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:33653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:32789
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:38.551788 7690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:38.552098 7690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:38.552177 7690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:38.556499 7690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:28:38.556591 7690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:38.556627 7690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:38.556679 7690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:28:38.556732 7690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:28:38.563035 7690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:33653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:32789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:32789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:38.564584 7690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:38.565985 7690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:28:38.578644 7690 server_base.cc:1034] running on GCE node
W20260808 14:28:38.579066 7695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:38.579087 7698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:38.579408 7696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:38.580228 7690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:38.582058 7690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:38.583252 7690 hybrid_clock.cc:648] HybridClock initialized: now 1786199318583207 us; error 78 us; skew 500 ppm
Aug 08 14:28:38 dist-test-slave-w206 krb5kdc[7676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199318, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:38.586254 7690 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:28:38.587517 7690 webserver.cc:460] Webserver started at http://127.7.125.126:45651/ using document root <none> and password file <none>
I20260808 14:28:38.588452 7690 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:38.588522 7690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:38.588876 7690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:38.592139 7690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "1bafa8921628498a8716143352f0ae86"
format_stamp: "Formatted at 2026-08-08 14:28:38 on dist-test-slave-w206"
server_key: "22280ee878385e748170dce6ee8b75b1"
server_key_iv: "f72d7aa155ce97616a95e7f6f024aad4"
server_key_version: "encryptionkey@0"
I20260808 14:28:38.592689 7690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "1bafa8921628498a8716143352f0ae86"
format_stamp: "Formatted at 2026-08-08 14:28:38 on dist-test-slave-w206"
server_key: "22280ee878385e748170dce6ee8b75b1"
server_key_iv: "f72d7aa155ce97616a95e7f6f024aad4"
server_key_version: "encryptionkey@0"
I20260808 14:28:38.598083 7690 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 14:28:38.601809 7706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:38.603533 7690 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:28:38.603700 7690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "1bafa8921628498a8716143352f0ae86"
format_stamp: "Formatted at 2026-08-08 14:28:38 on dist-test-slave-w206"
server_key: "22280ee878385e748170dce6ee8b75b1"
server_key_iv: "f72d7aa155ce97616a95e7f6f024aad4"
server_key_version: "encryptionkey@0"
I20260808 14:28:38.603803 7690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:38.615890 7690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:38.619362 7690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:38.619553 7690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:38.629608 7690 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:32789
I20260808 14:28:38.629674 7762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:32789 every 8 connection(s)
I20260808 14:28:38.631072 7690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:28:38.632934 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 7690
I20260808 14:28:38.633106 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:28:38.635501 7669 external_mini_cluster.cc:1428] Setting key 080224c25212745eab5af6ccc4a15f9b
I20260808 14:28:38.637699 7763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:38.649226 7763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86: Bootstrap starting.
I20260808 14:28:38.652992 7763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:38.654186 7763 log.cc:824] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86: Log is configured to *not* fsync() on all Append() calls
I20260808 14:28:38.656739 7763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86: No bootstrap required, opened a new log
Aug 08 14:28:38 dist-test-slave-w206 krb5kdc[7676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:38.661448 7763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bafa8921628498a8716143352f0ae86" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 32789 } }
I20260808 14:28:38.661749 7763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:38.661875 7763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bafa8921628498a8716143352f0ae86, State: Initialized, Role: FOLLOWER
I20260808 14:28:38.662729 7763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bafa8921628498a8716143352f0ae86" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 32789 } }
I20260808 14:28:38.662992 7763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:28:38.663125 7763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:28:38.663276 7763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:38.665616 7765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:38.641631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48622 (local address 127.7.125.126:32789)
0808 14:28:38.642240 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:38.642262 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:38.643535 (+ 1273us) server_negotiation.cc:407] Connection header received
0808 14:28:38.648771 (+ 5236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:38.648789 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:38.649224 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:38.649608 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:38.652034 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:38.653155 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:38.654411 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:38.655038 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:38.655216 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:38.659361 (+ 4145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:38.659406 (+ 45us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:38.659418 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:38.659446 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:38.661578 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:38.662316 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:38.662322 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:38.662328 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:38.662516 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:38.662898 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:38.662901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:38.662903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:38.663641 (+ 738us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:38.663836 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:38.664419 (+ 583us) server_negotiation.cc:299] Negotiation successful
0808 14:28:38.664642 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":169,"threads_started":1}
I20260808 14:28:38.735530 7763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bafa8921628498a8716143352f0ae86" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 32789 } }
I20260808 14:28:38.736340 7763 leader_election.cc:302] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1bafa8921628498a8716143352f0ae86; no voters:
I20260808 14:28:38.736860 7763 leader_election.cc:288] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:28:38.737079 7768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:28:38.737351 7768 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86 [term 1 LEADER]: Becoming Leader. State: Replica: 1bafa8921628498a8716143352f0ae86, State: Running, Role: LEADER
I20260808 14:28:38.737934 7768 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bafa8921628498a8716143352f0ae86" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 32789 } }
I20260808 14:28:38.738842 7763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:28:38.741597 7770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1bafa8921628498a8716143352f0ae86. Latest consensus state: current_term: 1 leader_uuid: "1bafa8921628498a8716143352f0ae86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bafa8921628498a8716143352f0ae86" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 32789 } } }
I20260808 14:28:38.741696 7769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1bafa8921628498a8716143352f0ae86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bafa8921628498a8716143352f0ae86" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 32789 } } }
I20260808 14:28:38.741781 7770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86 [sys.catalog]: This master's current role is: LEADER
I20260808 14:28:38.741781 7769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86 [sys.catalog]: This master's current role is: LEADER
W20260808 14:28:38.746402 7782 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:28:38.746531 7782 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:28:38.746616 7783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:28:38.747643 7783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:28:38.755865 7783 catalog_manager.cc:1347] Generated new cluster ID: 50db5bbe90e24837950c6cbf1c5ec31b
I20260808 14:28:38.755968 7783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:28:38.775048 7783 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:28:38.776510 7783 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:28:38.784193 7783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1bafa8921628498a8716143352f0ae86: Generated new TSK 0
I20260808 14:28:38.785241 7783 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.125.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
W20260808 14:28:39.717482 7669 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.7.125.65: real 0.918s user 0.005s sys 0.000s
I20260808 14:28:39.718065 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:32789
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:33653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:39.882762 7790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:39.883107 7790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:39.883226 7790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:39.887660 7790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:39.887722 7790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:39.887825 7790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:28:39.893822 7790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:33653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:32789
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:39.895104 7790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:39.896234 7790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:28:39.907840 7796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:39.907935 7797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:39.908097 7799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:39.908250 7790 server_base.cc:1034] running on GCE node
I20260808 14:28:39.908649 7790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:39.909246 7790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:39.910482 7790 hybrid_clock.cc:648] HybridClock initialized: now 1786199319910458 us; error 44 us; skew 500 ppm
Aug 08 14:28:39 dist-test-slave-w206 krb5kdc[7676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199319, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:39.913980 7790 init.cc:377] Logged in from keytab as kudu/127.7.125.65@KRBTEST.COM (short username kudu)
I20260808 14:28:39.914892 7790 webserver.cc:460] Webserver started at http://127.7.125.65:35815/ using document root <none> and password file <none>
I20260808 14:28:39.915513 7790 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:39.915709 7790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:39.916030 7790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:39.918434 7790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "0d755c1a74ec4a08b222a8a044253e7e"
format_stamp: "Formatted at 2026-08-08 14:28:39 on dist-test-slave-w206"
server_key: "8312197f60a68fddfd957686c3284475"
server_key_iv: "c9179040940e02921043fc349b428900"
server_key_version: "encryptionkey@0"
I20260808 14:28:39.919185 7790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "0d755c1a74ec4a08b222a8a044253e7e"
format_stamp: "Formatted at 2026-08-08 14:28:39 on dist-test-slave-w206"
server_key: "8312197f60a68fddfd957686c3284475"
server_key_iv: "c9179040940e02921043fc349b428900"
server_key_version: "encryptionkey@0"
I20260808 14:28:39.923554 7790 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 14:28:39.926868 7806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:39.928452 7790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:28:39.928574 7790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "0d755c1a74ec4a08b222a8a044253e7e"
format_stamp: "Formatted at 2026-08-08 14:28:39 on dist-test-slave-w206"
server_key: "8312197f60a68fddfd957686c3284475"
server_key_iv: "c9179040940e02921043fc349b428900"
server_key_version: "encryptionkey@0"
I20260808 14:28:39.928689 7790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:39.940196 7790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:39.943859 7790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:39.944103 7790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:39.945007 7790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:28:39.946365 7790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:28:39.946419 7790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:39.946461 7790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:28:39.946482 7790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:39.965953 7790 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:44977
I20260808 14:28:39.966028 7924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:44977 every 8 connection(s)
I20260808 14:28:39.967042 7790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:28:39.976994 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 7790
I20260808 14:28:39.977089 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:28:39.977347 7669 external_mini_cluster.cc:1428] Setting key a93833554a8ca5f7d7bf5cace9026e5f
Aug 08 14:28:39 dist-test-slave-w206 krb5kdc[7676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199319, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:39.984819 7927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:39.969604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:44971 (local address 127.7.125.126:32789)
0808 14:28:39.969948 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:39.969952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:39.970854 (+ 902us) server_negotiation.cc:407] Connection header received
0808 14:28:39.972832 (+ 1978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:39.972836 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:39.972899 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:39.973024 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:39.974429 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:39.975021 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:39.976262 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:39.976505 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:39.976618 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:39.979711 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:39.979718 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:39.979724 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:39.979758 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:39.982074 (+ 2316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:39.982768 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:39.982772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:39.982773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:39.982866 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:39.983326 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:39.983333 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:39.983337 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:39.983632 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:39.983752 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:39.984329 (+ 577us) server_negotiation.cc:299] Negotiation successful
0808 14:28:39.984498 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":148,"threads_started":1}
I20260808 14:28:39.985453 7928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:39.970049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:32789 (local address 127.7.125.65:44971)
0808 14:28:39.970690 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:39.970735 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:39.972606 (+ 1871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:39.973205 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:39.973213 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:39.973712 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:39.974260 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:39.974275 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:39.975155 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:39.975165 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:39.976126 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:39.976137 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:39.976673 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:39.976677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:39.976939 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:39.977704 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:39.977720 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:39.979543 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:39.982209 (+ 2666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:39.982218 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:39.982236 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:39.982645 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:39.983001 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:39.983005 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:39.983007 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:39.983208 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:39.983747 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:39.983752 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:39.983994 (+ 242us) client_negotiation.cc:769] Sending connection context
0808 14:28:39.984164 (+ 170us) client_negotiation.cc:240] Negotiation successful
0808 14:28:39.984456 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":199,"threads_started":1}
I20260808 14:28:39.986905 7925 heartbeater.cc:347] Connected to a master server at 127.7.125.126:32789
I20260808 14:28:39.987207 7925 heartbeater.cc:464] Registering TS with master...
I20260808 14:28:39.987700 7925 heartbeater.cc:511] Master 127.7.125.126:32789 requested a full tablet report, sending...
I20260808 14:28:39.990235 7725 ts_manager.cc:194] Registered new tserver with Master: 0d755c1a74ec4a08b222a8a044253e7e (127.7.125.65:44977)
WARNING: no policy specified for kudu/127.7.125.66@KRBTEST.COM; defaulting to no policy
I20260808 14:28:39.992599 7725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:44971
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
I20260808 14:28:40.014616 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:32789
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:33653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:40.128290 7931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:40.128551 7931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:40.128594 7931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:40.132717 7931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:40.132772 7931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:40.132831 7931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:28:40.138135 7931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:33653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:32789
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:40.139256 7931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:40.140389 7931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:28:40.150666 7939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:40.150765 7936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:40.150673 7937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:40.150878 7931 server_base.cc:1034] running on GCE node
I20260808 14:28:40.151290 7931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:40.151968 7931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:40.153133 7931 hybrid_clock.cc:648] HybridClock initialized: now 1786199320153098 us; error 45 us; skew 500 ppm
Aug 08 14:28:40 dist-test-slave-w206 krb5kdc[7676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199320, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:40.156001 7931 init.cc:377] Logged in from keytab as kudu/127.7.125.66@KRBTEST.COM (short username kudu)
I20260808 14:28:40.156883 7931 webserver.cc:460] Webserver started at http://127.7.125.66:41957/ using document root <none> and password file <none>
I20260808 14:28:40.157496 7931 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:40.157593 7931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:40.157830 7931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:40.160113 7931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "ca7b4c91ada64453892f99dc48868591"
format_stamp: "Formatted at 2026-08-08 14:28:40 on dist-test-slave-w206"
server_key: "f59b43b634565837d45881e43452e18c"
server_key_iv: "a56b6a25915c9cb89c0b7c9ab45f5b59"
server_key_version: "encryptionkey@0"
I20260808 14:28:40.160709 7931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "ca7b4c91ada64453892f99dc48868591"
format_stamp: "Formatted at 2026-08-08 14:28:40 on dist-test-slave-w206"
server_key: "f59b43b634565837d45881e43452e18c"
server_key_iv: "a56b6a25915c9cb89c0b7c9ab45f5b59"
server_key_version: "encryptionkey@0"
I20260808 14:28:40.164793 7931 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:28:40.167284 7946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:40.168571 7931 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:28:40.168712 7931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "ca7b4c91ada64453892f99dc48868591"
format_stamp: "Formatted at 2026-08-08 14:28:40 on dist-test-slave-w206"
server_key: "f59b43b634565837d45881e43452e18c"
server_key_iv: "a56b6a25915c9cb89c0b7c9ab45f5b59"
server_key_version: "encryptionkey@0"
I20260808 14:28:40.168833 7931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:40.181798 7931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:40.184726 7931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:40.184937 7931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:40.185537 7931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:28:40.186359 7931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:28:40.186455 7931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:40.186529 7931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:28:40.186573 7931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:40.195768 7931 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:37623
I20260808 14:28:40.195796 8065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:37623 every 8 connection(s)
I20260808 14:28:40.196661 7931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:28:40.202569 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 7931
I20260808 14:28:40.202721 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:28:40.203071 7669 external_mini_cluster.cc:1428] Setting key dfb1699c1e7c721dfe72abce1e78cba6
Aug 08 14:28:40 dist-test-slave-w206 krb5kdc[7676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199320, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:40.211570 7927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:40.198612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:54621 (local address 127.7.125.126:32789)
0808 14:28:40.198754 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:40.198758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:40.199626 (+ 868us) server_negotiation.cc:407] Connection header received
0808 14:28:40.201247 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:40.201250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:40.201313 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:40.201420 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:40.202620 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:40.203386 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:40.204751 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:40.204995 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:40.205105 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:40.207790 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:40.207794 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:40.207796 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:40.207830 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:40.209397 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:40.209930 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:40.209934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:40.209936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:40.210024 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:40.210376 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:40.210380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:40.210382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:40.210569 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:40.210677 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:40.211230 (+ 553us) server_negotiation.cc:299] Negotiation successful
0808 14:28:40.211368 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 14:28:40.212369 8068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:40.198962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:32789 (local address 127.7.125.66:54621)
0808 14:28:40.199475 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:40.199517 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:40.201000 (+ 1483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:40.201597 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:40.201603 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:40.201969 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:40.202467 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:40.202481 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:40.203516 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:40.203524 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:40.204617 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:40.204627 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:40.205145 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:40.205149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:40.205321 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:40.206004 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:40.206020 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:40.207627 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:40.209532 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:40.209538 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:40.209554 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:40.209819 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:40.210126 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:40.210130 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:40.210132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:40.210278 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:40.210678 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:40.210685 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:40.210936 (+ 251us) client_negotiation.cc:769] Sending connection context
0808 14:28:40.211190 (+ 254us) client_negotiation.cc:240] Negotiation successful
0808 14:28:40.211525 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":111,"threads_started":1}
WARNING: no policy specified for kudu/127.7.125.67@KRBTEST.COM; defaulting to no policy
I20260808 14:28:40.213660 8066 heartbeater.cc:347] Connected to a master server at 127.7.125.126:32789
I20260808 14:28:40.213934 8066 heartbeater.cc:464] Registering TS with master...
I20260808 14:28:40.214501 8066 heartbeater.cc:511] Master 127.7.125.126:32789 requested a full tablet report, sending...
I20260808 14:28:40.215585 7725 ts_manager.cc:194] Registered new tserver with Master: ca7b4c91ada64453892f99dc48868591 (127.7.125.66:37623)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.67@KRBTEST.COM" created.
I20260808 14:28:40.216379 7725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:54621
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
I20260808 14:28:40.234021 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:32789
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:33653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:40.345387 8071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:40.345660 8071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:40.345706 8071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:40.349933 8071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:40.350004 8071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:40.350065 8071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:28:40.355496 8071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:33653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:32789
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:40.356527 8071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:40.357556 8071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:28:40.367156 8076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:40.367143 8079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:40.367272 8071 server_base.cc:1034] running on GCE node
W20260808 14:28:40.367143 8077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:40.367779 8071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:40.368445 8071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:40.369625 8071 hybrid_clock.cc:648] HybridClock initialized: now 1786199320369612 us; error 24 us; skew 500 ppm
Aug 08 14:28:40 dist-test-slave-w206 krb5kdc[7676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199320, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:40.372457 8071 init.cc:377] Logged in from keytab as kudu/127.7.125.67@KRBTEST.COM (short username kudu)
I20260808 14:28:40.373322 8071 webserver.cc:460] Webserver started at http://127.7.125.67:46851/ using document root <none> and password file <none>
I20260808 14:28:40.373890 8071 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:40.373981 8071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:40.374213 8071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:40.376267 8071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "8c076b808be1449d872b1b6ae8625f5c"
format_stamp: "Formatted at 2026-08-08 14:28:40 on dist-test-slave-w206"
server_key: "8a89886bc4a32441ec09e156baf480b9"
server_key_iv: "00bb84a015f35c71448253360c813b0b"
server_key_version: "encryptionkey@0"
I20260808 14:28:40.376881 8071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "8c076b808be1449d872b1b6ae8625f5c"
format_stamp: "Formatted at 2026-08-08 14:28:40 on dist-test-slave-w206"
server_key: "8a89886bc4a32441ec09e156baf480b9"
server_key_iv: "00bb84a015f35c71448253360c813b0b"
server_key_version: "encryptionkey@0"
I20260808 14:28:40.380698 8071 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 14:28:40.383167 8087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:40.384387 8071 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 14:28:40.384507 8071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "8c076b808be1449d872b1b6ae8625f5c"
format_stamp: "Formatted at 2026-08-08 14:28:40 on dist-test-slave-w206"
server_key: "8a89886bc4a32441ec09e156baf480b9"
server_key_iv: "00bb84a015f35c71448253360c813b0b"
server_key_version: "encryptionkey@0"
I20260808 14:28:40.384611 8071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:40.400358 8071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:40.403388 8071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:40.403592 8071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:40.404194 8071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:28:40.405019 8071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:28:40.405066 8071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:40.405138 8071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:28:40.405174 8071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:40.415043 8071 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:36611
I20260808 14:28:40.415108 8203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:36611 every 8 connection(s)
I20260808 14:28:40.415961 8071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:28:40.421842 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 8071
I20260808 14:28:40.421947 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:28:40.422230 7669 external_mini_cluster.cc:1428] Setting key a0a3a241ee890e6bc623cb7c90deaa93
Aug 08 14:28:40 dist-test-slave-w206 krb5kdc[7676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199320, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:40.429754 7927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:40.417779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:42651 (local address 127.7.125.126:32789)
0808 14:28:40.417919 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:40.417923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:40.418735 (+ 812us) server_negotiation.cc:407] Connection header received
0808 14:28:40.420152 (+ 1417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:40.420155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:40.420215 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:40.420321 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:40.421457 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:40.422095 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:40.423494 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:40.423756 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:40.423864 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:40.426644 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:40.426648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:40.426650 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:40.426674 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:40.427994 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:40.428468 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:40.428470 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:40.428471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:40.428549 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:40.428848 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:40.428850 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:40.428851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:40.429022 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:40.429097 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:40.429542 (+ 445us) server_negotiation.cc:299] Negotiation successful
0808 14:28:40.429637 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 14:28:40.430603 8206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:40.418138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:32789 (local address 127.7.125.67:42651)
0808 14:28:40.418591 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:40.418634 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:40.419943 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:40.420490 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:40.420497 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:40.420841 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:40.421307 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:40.421320 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:40.422218 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:40.422227 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:40.423359 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:40.423370 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:40.423933 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:40.423938 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:40.424186 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:40.424914 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:40.424935 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:40.426487 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:40.428097 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:40.428101 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:40.428113 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:40.428366 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:40.428631 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:40.428633 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:40.428635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:40.428765 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:40.429108 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:40.429113 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:40.429330 (+ 217us) client_negotiation.cc:769] Sending connection context
0808 14:28:40.429492 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 14:28:40.429737 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":123,"threads_started":1}
I20260808 14:28:40.431778 8204 heartbeater.cc:347] Connected to a master server at 127.7.125.126:32789
I20260808 14:28:40.432101 8204 heartbeater.cc:464] Registering TS with master...
I20260808 14:28:40.432637 8204 heartbeater.cc:511] Master 127.7.125.126:32789 requested a full tablet report, sending...
I20260808 14:28:40.433743 7725 ts_manager.cc:194] Registered new tserver with Master: 8c076b808be1449d872b1b6ae8625f5c (127.7.125.67:36611)
I20260808 14:28:40.434383 7725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:42651
I20260808 14:28:40.436823 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 14:28:40 dist-test-slave-w206 krb5kdc[7676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199320, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 14:28:40 dist-test-slave-w206 krb5kdc[7676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199320, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.125.65@KRBTEST.COM
I20260808 14:28:40.465873 8211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:40.455852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34176 (local address 127.7.125.65:44977)
0808 14:28:40.456158 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:40.456172 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:40.456197 (+ 25us) server_negotiation.cc:407] Connection header received
0808 14:28:40.456422 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:40.456430 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:40.456640 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:40.456795 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:40.457487 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:40.458396 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:40.459358 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:40.459741 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:40.459833 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:40.462491 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:40.462500 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:40.462514 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:40.462536 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:40.464192 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:40.464608 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:40.464613 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:40.464614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:40.464697 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:40.464933 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:40.464935 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:40.464936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:40.465172 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:40.465382 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:40.465491 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 14:28:40.465643 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":113,"threads_started":1}
W20260808 14:28:40.467015 7835 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34176
Aug 08 14:28:40 dist-test-slave-w206 krb5kdc[7676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 14:28:40 dist-test-slave-w206 krb5kdc[7676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.65@KRBTEST.COM
I20260808 14:28:40.494436 8211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:40.486174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34192 (local address 127.7.125.65:44977)
0808 14:28:40.486351 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:40.486356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:40.486519 (+ 163us) server_negotiation.cc:407] Connection header received
0808 14:28:40.486704 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:40.486707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:40.486754 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:40.486841 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:40.487617 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:40.488267 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:40.489099 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:40.489414 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:40.489522 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:40.491814 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:40.491817 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:40.491819 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:40.491841 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:40.493077 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:40.493453 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:40.493456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:40.493457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:40.493529 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:40.493749 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:40.493752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:40.493753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:40.493911 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:40.494008 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:40.494218 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 14:28:40.494313 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:28:40.495640 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 7790
I20260808 14:28:40.503598 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 7931
I20260808 14:28:40.510185 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 8071
I20260808 14:28:40.518163 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 7690
2026-08-08T14:28:40Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3617 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:28:40 dist-test-slave-w206 krb5kdc[8218](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:28:40 dist-test-slave-w206 krb5kdc[8218](info): set up 2 sockets
Aug 08 14:28:40 dist-test-slave-w206 krb5kdc[8218](info): commencing operation
krb5kdc: starting...
W20260808 14:28:41.591214 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:28:41 dist-test-slave-w206 krb5kdc[8218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:28:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:28:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:28:41.719056 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:43777
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:32813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:43777
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:41.830410 8231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:41.830725 8231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:41.830802 8231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:41.834797 8231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:28:41.834903 8231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:41.834939 8231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:41.834975 8231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:28:41.835029 8231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:28:41.840288 8231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:32813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:43777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:43777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:41.841378 8231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:41.842458 8231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:28:41.851210 8231 server_base.cc:1034] running on GCE node
W20260808 14:28:41.851277 8237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:41.851218 8239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:41.851191 8236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:41.851961 8231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:41.853068 8231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:41.854267 8231 hybrid_clock.cc:648] HybridClock initialized: now 1786199321854248 us; error 31 us; skew 500 ppm
Aug 08 14:28:41 dist-test-slave-w206 krb5kdc[8218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199321, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:41.857081 8231 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:28:41.857965 8231 webserver.cc:460] Webserver started at http://127.7.125.126:43267/ using document root <none> and password file <none>
I20260808 14:28:41.858531 8231 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:41.858632 8231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:41.858861 8231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:41.860937 8231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "7a020eae4a704e32a888e636b8c1fa9a"
format_stamp: "Formatted at 2026-08-08 14:28:41 on dist-test-slave-w206"
server_key: "7d8710e257e5ad766485f8857fedcf68"
server_key_iv: "5e00e2c95e22572808f3874e38c23379"
server_key_version: "encryptionkey@0"
I20260808 14:28:41.861502 8231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "7a020eae4a704e32a888e636b8c1fa9a"
format_stamp: "Formatted at 2026-08-08 14:28:41 on dist-test-slave-w206"
server_key: "7d8710e257e5ad766485f8857fedcf68"
server_key_iv: "5e00e2c95e22572808f3874e38c23379"
server_key_version: "encryptionkey@0"
I20260808 14:28:41.865267 8231 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 14:28:41.867638 8246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:41.868805 8231 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:28:41.868947 8231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "7a020eae4a704e32a888e636b8c1fa9a"
format_stamp: "Formatted at 2026-08-08 14:28:41 on dist-test-slave-w206"
server_key: "7d8710e257e5ad766485f8857fedcf68"
server_key_iv: "5e00e2c95e22572808f3874e38c23379"
server_key_version: "encryptionkey@0"
I20260808 14:28:41.869059 8231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:41.890144 8231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:41.893162 8231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:41.893384 8231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:41.900955 8231 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:43777
I20260808 14:28:41.900982 8302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:43777 every 8 connection(s)
I20260808 14:28:41.901978 8231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:28:41.904690 8303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:41.905771 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 8231
I20260808 14:28:41.905907 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:28:41.906164 7669 external_mini_cluster.cc:1428] Setting key 57ad3ac87dcf875c4eafd2af55c7e542
I20260808 14:28:41.911434 8303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a020eae4a704e32a888e636b8c1fa9a: Bootstrap starting.
I20260808 14:28:41.914063 8303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a020eae4a704e32a888e636b8c1fa9a: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:41.914835 8303 log.cc:824] T 00000000000000000000000000000000 P 7a020eae4a704e32a888e636b8c1fa9a: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:28:41 dist-test-slave-w206 krb5kdc[8218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:41.917146 8303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a020eae4a704e32a888e636b8c1fa9a: No bootstrap required, opened a new log
I20260808 14:28:41.921686 8306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:41.907626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46610 (local address 127.7.125.126:43777)
0808 14:28:41.908063 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:41.908082 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:41.908125 (+ 43us) server_negotiation.cc:407] Connection header received
0808 14:28:41.908958 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:41.908980 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:41.909389 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:41.909755 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:41.910449 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:41.911881 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:41.912702 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:41.913378 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:41.913537 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:41.916269 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:41.916278 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:41.916298 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:41.916330 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:41.918413 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:41.918835 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:41.918840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:41.918847 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:41.919027 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:41.919302 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:41.919307 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:41.919309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:41.920014 (+ 705us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:41.920273 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:41.920424 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 14:28:41.920722 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":114,"threads_started":1}
I20260808 14:28:41.922173 8303 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a020eae4a704e32a888e636b8c1fa9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a020eae4a704e32a888e636b8c1fa9a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43777 } }
I20260808 14:28:41.922382 8303 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a020eae4a704e32a888e636b8c1fa9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:41.922446 8303 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a020eae4a704e32a888e636b8c1fa9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a020eae4a704e32a888e636b8c1fa9a, State: Initialized, Role: FOLLOWER
I20260808 14:28:41.922906 8303 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a020eae4a704e32a888e636b8c1fa9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a020eae4a704e32a888e636b8c1fa9a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43777 } }
I20260808 14:28:41.923081 8303 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a020eae4a704e32a888e636b8c1fa9a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:28:41.923171 8303 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a020eae4a704e32a888e636b8c1fa9a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:28:41.923270 8303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a020eae4a704e32a888e636b8c1fa9a [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:41.924183 8303 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a020eae4a704e32a888e636b8c1fa9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a020eae4a704e32a888e636b8c1fa9a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43777 } }
I20260808 14:28:41.924525 8303 leader_election.cc:302] T 00000000000000000000000000000000 P 7a020eae4a704e32a888e636b8c1fa9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a020eae4a704e32a888e636b8c1fa9a; no voters:
I20260808 14:28:41.924835 8303 leader_election.cc:288] T 00000000000000000000000000000000 P 7a020eae4a704e32a888e636b8c1fa9a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:28:41.925074 8308 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a020eae4a704e32a888e636b8c1fa9a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:28:41.925384 8308 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a020eae4a704e32a888e636b8c1fa9a [term 1 LEADER]: Becoming Leader. State: Replica: 7a020eae4a704e32a888e636b8c1fa9a, State: Running, Role: LEADER
I20260808 14:28:41.925668 8308 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a020eae4a704e32a888e636b8c1fa9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a020eae4a704e32a888e636b8c1fa9a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43777 } }
I20260808 14:28:41.926096 8303 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a020eae4a704e32a888e636b8c1fa9a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:28:41.927382 8310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a020eae4a704e32a888e636b8c1fa9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a020eae4a704e32a888e636b8c1fa9a. Latest consensus state: current_term: 1 leader_uuid: "7a020eae4a704e32a888e636b8c1fa9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a020eae4a704e32a888e636b8c1fa9a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43777 } } }
I20260808 14:28:41.927382 8309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a020eae4a704e32a888e636b8c1fa9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a020eae4a704e32a888e636b8c1fa9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a020eae4a704e32a888e636b8c1fa9a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43777 } } }
I20260808 14:28:41.927605 8309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a020eae4a704e32a888e636b8c1fa9a [sys.catalog]: This master's current role is: LEADER
I20260808 14:28:41.927568 8310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a020eae4a704e32a888e636b8c1fa9a [sys.catalog]: This master's current role is: LEADER
I20260808 14:28:41.928625 8318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:28:41.931763 8318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:28:41.937470 8318 catalog_manager.cc:1347] Generated new cluster ID: 4a9e124df2cb4447885e4426bf51a697
I20260808 14:28:41.937533 8318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:28:41.949884 8318 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:28:41.951179 8318 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:28:41.959532 8318 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a020eae4a704e32a888e636b8c1fa9a: Generated new TSK 0
I20260808 14:28:41.960352 8318 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.125.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
I20260808 14:28:41.994091 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:43777
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:32813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:42.106438 8330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:42.106737 8330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:42.106817 8330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:42.110847 8330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:42.110948 8330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:42.111068 8330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:28:42.116272 8330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:32813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:43777
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:42.117362 8330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:42.118434 8330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:28:42.128333 8338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:42.128490 8330 server_base.cc:1034] running on GCE node
W20260808 14:28:42.128203 8336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:42.128203 8335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:42.129010 8330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:42.129645 8330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:42.130807 8330 hybrid_clock.cc:648] HybridClock initialized: now 1786199322130787 us; error 32 us; skew 500 ppm
Aug 08 14:28:42 dist-test-slave-w206 krb5kdc[8218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199322, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:42.138342 8330 init.cc:377] Logged in from keytab as kudu/127.7.125.65@KRBTEST.COM (short username kudu)
I20260808 14:28:42.139328 8330 webserver.cc:460] Webserver started at http://127.7.125.65:41443/ using document root <none> and password file <none>
I20260808 14:28:42.139907 8330 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:42.140012 8330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:42.140240 8330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:42.142341 8330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "65cbb2a06bde43c8bc0d3d93b26a8e76"
format_stamp: "Formatted at 2026-08-08 14:28:42 on dist-test-slave-w206"
server_key: "de154d139474e473b7af513fbbae0db0"
server_key_iv: "5cf39c426f5f0d3012caee1ca0b4afd1"
server_key_version: "encryptionkey@0"
I20260808 14:28:42.142964 8330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "65cbb2a06bde43c8bc0d3d93b26a8e76"
format_stamp: "Formatted at 2026-08-08 14:28:42 on dist-test-slave-w206"
server_key: "de154d139474e473b7af513fbbae0db0"
server_key_iv: "5cf39c426f5f0d3012caee1ca0b4afd1"
server_key_version: "encryptionkey@0"
I20260808 14:28:42.147032 8330 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:28:42.149417 8345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:42.150560 8330 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:28:42.150698 8330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "65cbb2a06bde43c8bc0d3d93b26a8e76"
format_stamp: "Formatted at 2026-08-08 14:28:42 on dist-test-slave-w206"
server_key: "de154d139474e473b7af513fbbae0db0"
server_key_iv: "5cf39c426f5f0d3012caee1ca0b4afd1"
server_key_version: "encryptionkey@0"
I20260808 14:28:42.150810 8330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:42.167765 8330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:42.170740 8330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:42.170967 8330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:42.171635 8330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:28:42.172537 8330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:28:42.172607 8330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:42.172677 8330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:28:42.172729 8330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:42.182149 8330 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:32859
I20260808 14:28:42.182169 8463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:32859 every 8 connection(s)
I20260808 14:28:42.183166 8330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:28:42.191357 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 8330
I20260808 14:28:42.191510 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:28:42.191780 7669 external_mini_cluster.cc:1428] Setting key f43f6739be5ece599d857b159184279a
Aug 08 14:28:42 dist-test-slave-w206 krb5kdc[8218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199322, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:42.197012 8306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:42.184925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:37149 (local address 127.7.125.126:43777)
0808 14:28:42.185099 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:42.185104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:42.185953 (+ 849us) server_negotiation.cc:407] Connection header received
0808 14:28:42.187311 (+ 1358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:42.187314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:42.187363 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:42.187448 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:42.188588 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.189119 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.190204 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.190467 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.190550 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:42.193395 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:42.193405 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:42.193408 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:42.193431 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:42.194888 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:42.195418 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:42.195423 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:42.195425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:42.195510 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:42.195867 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:42.195870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:42.195872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:42.196060 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:42.196169 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:42.196764 (+ 595us) server_negotiation.cc:299] Negotiation successful
0808 14:28:42.196877 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 14:28:42.198237 8467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:42.185322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43777 (local address 127.7.125.65:37149)
0808 14:28:42.185779 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:42.185834 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:42.187113 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:42.187610 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:42.187616 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:42.187989 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:42.188461 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:42.188474 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.189227 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.189230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:42.190101 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:42.190109 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.190602 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.190605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:42.190741 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:42.191456 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:42.191473 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:42.193235 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:42.195023 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:42.195030 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:42.195046 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:42.195313 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:42.195607 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:42.195611 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:42.195613 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:42.195758 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:42.196167 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:42.196174 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:42.196480 (+ 306us) client_negotiation.cc:769] Sending connection context
0808 14:28:42.196682 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 14:28:42.197054 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":120,"threads_started":1}
I20260808 14:28:42.199527 8464 heartbeater.cc:347] Connected to a master server at 127.7.125.126:43777
I20260808 14:28:42.199848 8464 heartbeater.cc:464] Registering TS with master...
I20260808 14:28:42.200482 8464 heartbeater.cc:511] Master 127.7.125.126:43777 requested a full tablet report, sending...
I20260808 14:28:42.202006 8265 ts_manager.cc:194] Registered new tserver with Master: 65cbb2a06bde43c8bc0d3d93b26a8e76 (127.7.125.65:32859)
WARNING: no policy specified for kudu/127.7.125.66@KRBTEST.COM; defaulting to no policy
I20260808 14:28:42.203336 8265 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:37149
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
I20260808 14:28:42.222515 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:43777
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:32813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:42.359906 8470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:42.360216 8470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:42.360265 8470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:42.364351 8470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:42.364421 8470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:42.364483 8470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:28:42.369853 8470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:32813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:43777
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:42.370867 8470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:42.371837 8470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:28:42.381572 8476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:42.381624 8478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:42.381632 8470 server_base.cc:1034] running on GCE node
W20260808 14:28:42.381582 8475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:42.382198 8470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:42.382820 8470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:42.383985 8470 hybrid_clock.cc:648] HybridClock initialized: now 1786199322383975 us; error 32 us; skew 500 ppm
Aug 08 14:28:42 dist-test-slave-w206 krb5kdc[8218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199322, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:42.386740 8470 init.cc:377] Logged in from keytab as kudu/127.7.125.66@KRBTEST.COM (short username kudu)
I20260808 14:28:42.387661 8470 webserver.cc:460] Webserver started at http://127.7.125.66:44467/ using document root <none> and password file <none>
I20260808 14:28:42.388223 8470 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:42.388284 8470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:42.388558 8470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:42.390681 8470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "e425dd69ace248ee807e012e81d47ab6"
format_stamp: "Formatted at 2026-08-08 14:28:42 on dist-test-slave-w206"
server_key: "3c7d0062a5268fa5e733a60a03f46a7b"
server_key_iv: "d990db1d15d984b6455a290c73027ae6"
server_key_version: "encryptionkey@0"
I20260808 14:28:42.391294 8470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "e425dd69ace248ee807e012e81d47ab6"
format_stamp: "Formatted at 2026-08-08 14:28:42 on dist-test-slave-w206"
server_key: "3c7d0062a5268fa5e733a60a03f46a7b"
server_key_iv: "d990db1d15d984b6455a290c73027ae6"
server_key_version: "encryptionkey@0"
I20260808 14:28:42.395036 8470 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260808 14:28:42.397471 8485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:42.398692 8470 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 14:28:42.398820 8470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "e425dd69ace248ee807e012e81d47ab6"
format_stamp: "Formatted at 2026-08-08 14:28:42 on dist-test-slave-w206"
server_key: "3c7d0062a5268fa5e733a60a03f46a7b"
server_key_iv: "d990db1d15d984b6455a290c73027ae6"
server_key_version: "encryptionkey@0"
I20260808 14:28:42.398931 8470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:42.409616 8470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:42.412678 8470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:42.412889 8470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:42.413499 8470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:28:42.414330 8470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:28:42.414377 8470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:42.414454 8470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:28:42.414494 8470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:42.423882 8470 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:46815
I20260808 14:28:42.423911 8601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:46815 every 8 connection(s)
I20260808 14:28:42.424782 8470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:28:42.430883 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 8470
I20260808 14:28:42.431013 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:28:42.431274 7669 external_mini_cluster.cc:1428] Setting key 16572a488f0ca58fcd198c2029de4051
Aug 08 14:28:42 dist-test-slave-w206 krb5kdc[8218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199322, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:42.440536 8306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:42.426675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:33205 (local address 127.7.125.126:43777)
0808 14:28:42.426843 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:42.426848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:42.427815 (+ 967us) server_negotiation.cc:407] Connection header received
0808 14:28:42.429285 (+ 1470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:42.429288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:42.429333 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:42.429409 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:42.430618 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.431300 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.432804 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.433031 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.433112 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:42.436304 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:42.436310 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:42.436313 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:42.436344 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:42.438161 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:42.438680 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:42.438684 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:42.438686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:42.438785 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:42.439300 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:42.439302 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:42.439304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:42.439492 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:42.439573 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:42.440295 (+ 722us) server_negotiation.cc:299] Negotiation successful
0808 14:28:42.440412 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 14:28:42.441711 8604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:42.427152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43777 (local address 127.7.125.66:33205)
0808 14:28:42.427632 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:42.427676 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:42.429076 (+ 1400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:42.429576 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:42.429582 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:42.429952 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:42.430461 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:42.430473 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.431494 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.431498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:42.432659 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:42.432668 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.433186 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.433192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:42.433435 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:42.434117 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:42.434143 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:42.436131 (+ 1988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:42.438305 (+ 2174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:42.438310 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:42.438321 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:42.438575 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:42.438899 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:42.438904 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:42.438909 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:42.439134 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:42.439607 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:42.439616 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:42.439996 (+ 380us) client_negotiation.cc:769] Sending connection context
0808 14:28:42.440240 (+ 244us) client_negotiation.cc:240] Negotiation successful
0808 14:28:42.440610 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":128,"threads_started":1}
I20260808 14:28:42.443006 8602 heartbeater.cc:347] Connected to a master server at 127.7.125.126:43777
WARNING: no policy specified for kudu/127.7.125.67@KRBTEST.COM; defaulting to no policy
I20260808 14:28:42.443265 8602 heartbeater.cc:464] Registering TS with master...
I20260808 14:28:42.443773 8602 heartbeater.cc:511] Master 127.7.125.126:43777 requested a full tablet report, sending...
I20260808 14:28:42.444828 8265 ts_manager.cc:194] Registered new tserver with Master: e425dd69ace248ee807e012e81d47ab6 (127.7.125.66:46815)
I20260808 14:28:42.445530 8265 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:33205
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
I20260808 14:28:42.462445 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:43777
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:32813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:42.572564 8607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:42.572809 8607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:42.572852 8607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:42.576869 8607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:42.576927 8607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:42.577033 8607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:28:42.582286 8607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:32813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:43777
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:42.583458 8607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:42.584486 8607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:28:42.594223 8613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:42.594246 8615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:42.594260 8612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:42.594646 8607 server_base.cc:1034] running on GCE node
I20260808 14:28:42.595122 8607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:42.595697 8607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:42.596872 8607 hybrid_clock.cc:648] HybridClock initialized: now 1786199322596856 us; error 29 us; skew 500 ppm
Aug 08 14:28:42 dist-test-slave-w206 krb5kdc[8218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199322, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:42.599629 8607 init.cc:377] Logged in from keytab as kudu/127.7.125.67@KRBTEST.COM (short username kudu)
I20260808 14:28:42.600446 8607 webserver.cc:460] Webserver started at http://127.7.125.67:43517/ using document root <none> and password file <none>
I20260808 14:28:42.600950 8607 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:42.601006 8607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:42.601166 8607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:42.603427 8607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "2d38faa555414c018df92fcb0668afa9"
format_stamp: "Formatted at 2026-08-08 14:28:42 on dist-test-slave-w206"
server_key: "388f55e9073da752ed2e71e998d31534"
server_key_iv: "8e9a01c0606b616fb49e306ca7595131"
server_key_version: "encryptionkey@0"
I20260808 14:28:42.604084 8607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "2d38faa555414c018df92fcb0668afa9"
format_stamp: "Formatted at 2026-08-08 14:28:42 on dist-test-slave-w206"
server_key: "388f55e9073da752ed2e71e998d31534"
server_key_iv: "8e9a01c0606b616fb49e306ca7595131"
server_key_version: "encryptionkey@0"
I20260808 14:28:42.608011 8607 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:28:42.610368 8622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:42.611658 8607 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:28:42.611794 8607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "2d38faa555414c018df92fcb0668afa9"
format_stamp: "Formatted at 2026-08-08 14:28:42 on dist-test-slave-w206"
server_key: "388f55e9073da752ed2e71e998d31534"
server_key_iv: "8e9a01c0606b616fb49e306ca7595131"
server_key_version: "encryptionkey@0"
I20260808 14:28:42.611910 8607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:42.646340 8607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:42.649806 8607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:42.650017 8607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:42.650640 8607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:28:42.651563 8607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:28:42.651633 8607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:42.651705 8607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:28:42.651754 8607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:42.661003 8607 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:44553
I20260808 14:28:42.661028 8740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:44553 every 8 connection(s)
I20260808 14:28:42.661866 8607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:28:42.669359 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 8607
I20260808 14:28:42.669498 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:28:42.669762 7669 external_mini_cluster.cc:1428] Setting key 12a57fc32d178d78c7045bc3b2f93f1e
Aug 08 14:28:42 dist-test-slave-w206 krb5kdc[8218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199322, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:42.675544 8306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:42.663678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:56945 (local address 127.7.125.126:43777)
0808 14:28:42.663822 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:42.663826 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:42.664634 (+ 808us) server_negotiation.cc:407] Connection header received
0808 14:28:42.665980 (+ 1346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:42.665983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:42.666041 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:42.666142 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:42.667309 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.667856 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.668996 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.669225 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.669339 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:42.672358 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:42.672361 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:42.672364 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:42.672385 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:42.673685 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:42.674188 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:42.674191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:42.674192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:42.674266 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:42.674580 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:42.674582 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:42.674583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:42.674750 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:42.674835 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:42.675330 (+ 495us) server_negotiation.cc:299] Negotiation successful
0808 14:28:42.675429 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:28:42.676337 8743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:42.664054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43777 (local address 127.7.125.67:56945)
0808 14:28:42.664491 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:42.664532 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:42.665769 (+ 1237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:42.666306 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:42.666318 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:42.666653 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:42.667169 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:42.667184 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.667984 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.667994 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:42.668871 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:42.668880 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.669384 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.669388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:42.669627 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:42.670243 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:42.670265 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:42.672214 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:42.673825 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:42.673829 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:42.673841 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:42.674097 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:42.674348 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:42.674350 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:42.674351 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:42.674499 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:42.674835 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:42.674840 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:42.675123 (+ 283us) client_negotiation.cc:769] Sending connection context
0808 14:28:42.675280 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 14:28:42.675572 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":115,"threads_started":1}
I20260808 14:28:42.677407 8741 heartbeater.cc:347] Connected to a master server at 127.7.125.126:43777
I20260808 14:28:42.677608 8741 heartbeater.cc:464] Registering TS with master...
I20260808 14:28:42.678026 8741 heartbeater.cc:511] Master 127.7.125.126:43777 requested a full tablet report, sending...
I20260808 14:28:42.678941 8265 ts_manager.cc:194] Registered new tserver with Master: 2d38faa555414c018df92fcb0668afa9 (127.7.125.67:44553)
I20260808 14:28:42.679570 8265 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:56945
I20260808 14:28:42.683624 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:28:42.695227 8306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:42.687775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46624 (local address 127.7.125.126:43777)
0808 14:28:42.687946 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:42.687951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:42.688109 (+ 158us) server_negotiation.cc:407] Connection header received
0808 14:28:42.688261 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:42.688264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:42.688312 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:42.688395 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:42.689131 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.689623 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.690462 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.690679 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.690786 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:42.692216 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:42.692220 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:42.692223 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:42.692253 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:42.693453 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:42.693913 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:42.693919 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:42.693924 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:42.694003 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:42.694284 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:42.694288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:42.694289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:42.694516 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:42.694662 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:42.694910 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 14:28:42.695059 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 14:28:42.700830 8265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46624:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:28:42.704427 8265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:28:42.721125 8757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:42.715322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:44553 (local address 127.0.0.1:53570)
0808 14:28:42.716165 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:42.716188 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:42.716309 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:42.716981 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:42.716986 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:42.717005 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:28:42.717133 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:42.717139 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.718672 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.718676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:42.719758 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:42.719765 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.720543 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.720546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:42.720679 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:42.720720 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 14:28:42.720838 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 14:28:42.720958 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"mutex_wait_us":1,"thread_start_us":107,"threads_started":1}
I20260808 14:28:42.721174 8753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:42.714580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:46815 (local address 127.0.0.1:58486)
0808 14:28:42.715733 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:42.715790 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:42.715932 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:42.716399 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:42.716405 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:42.716427 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:28:42.716587 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:42.716593 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.718132 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.718135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:42.719511 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:42.719522 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.720850 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.720853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:42.720931 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:42.720950 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 14:28:42.721006 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 14:28:42.721062 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":986,"mutex_wait_us":28,"spinlock_wait_cycles":3968,"thread_start_us":76,"threads_started":1}
I20260808 14:28:42.721588 8758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:42.715630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53570 (local address 127.7.125.67:44553)
0808 14:28:42.716103 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:42.716118 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:42.716283 (+ 165us) server_negotiation.cc:407] Connection header received
0808 14:28:42.716465 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:42.716470 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:42.716665 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:42.716825 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:28:42.717261 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.718508 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.719874 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.720413 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.720539 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:42.720759 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:42.721336 (+ 577us) server_negotiation.cc:299] Negotiation successful
0808 14:28:42.721439 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":144,"threads_started":1}
I20260808 14:28:42.721774 8756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:42.715248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:32859 (local address 127.0.0.1:47020)
0808 14:28:42.715738 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:42.715791 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:42.715935 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:42.716810 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:42.716813 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:42.716836 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:28:42.716931 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:42.716936 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.718282 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.718285 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:42.719374 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:42.719384 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.720872 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.720877 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:42.720995 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:42.721021 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 14:28:42.721102 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 14:28:42.721180 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":1,"spinlock_wait_cycles":896,"thread_start_us":108,"threads_started":1}
I20260808 14:28:42.721913 8754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:42.714783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58486 (local address 127.7.125.66:46815)
0808 14:28:42.715946 (+ 1163us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:42.715957 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:42.715984 (+ 27us) server_negotiation.cc:407] Connection header received
0808 14:28:42.716064 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:42.716069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:42.716239 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:42.716408 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:28:42.716771 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.717984 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.719638 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.720489 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.720588 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:42.721583 (+ 995us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:42.721638 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 14:28:42.721739 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1065,"thread_start_us":122,"threads_started":1}
I20260808 14:28:42.722714 8755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:42.715242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47020 (local address 127.7.125.65:32859)
0808 14:28:42.715742 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:42.715750 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:42.716363 (+ 613us) server_negotiation.cc:407] Connection header received
0808 14:28:42.716458 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:42.716463 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:42.716666 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:42.716825 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:28:42.717068 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.718166 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.719506 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.720462 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.720588 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:42.720795 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:42.722493 (+ 1698us) server_negotiation.cc:299] Negotiation successful
0808 14:28:42.722576 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":120,"threads_started":1}
I20260808 14:28:42.724880 8672 tablet_service.cc:1467] Processing CreateTablet for tablet 2b72522f19ea48cd9c0ddb79c4ac3b88 (DEFAULT_TABLE table=test-table [id=22356b9158ea4d5b9b1ff7276c404b20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:28:42.724880 8534 tablet_service.cc:1467] Processing CreateTablet for tablet 2b72522f19ea48cd9c0ddb79c4ac3b88 (DEFAULT_TABLE table=test-table [id=22356b9158ea4d5b9b1ff7276c404b20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:28:42.725329 8395 tablet_service.cc:1467] Processing CreateTablet for tablet 2b72522f19ea48cd9c0ddb79c4ac3b88 (DEFAULT_TABLE table=test-table [id=22356b9158ea4d5b9b1ff7276c404b20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:28:42.726213 8534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b72522f19ea48cd9c0ddb79c4ac3b88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:42.726269 8395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b72522f19ea48cd9c0ddb79c4ac3b88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:42.728336 8672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b72522f19ea48cd9c0ddb79c4ac3b88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:42.733937 8759 tablet_bootstrap.cc:492] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6: Bootstrap starting.
I20260808 14:28:42.734894 8760 tablet_bootstrap.cc:492] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 2d38faa555414c018df92fcb0668afa9: Bootstrap starting.
I20260808 14:28:42.736797 8760 tablet_bootstrap.cc:654] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 2d38faa555414c018df92fcb0668afa9: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:42.736923 8761 tablet_bootstrap.cc:492] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 65cbb2a06bde43c8bc0d3d93b26a8e76: Bootstrap starting.
I20260808 14:28:42.737221 8759 tablet_bootstrap.cc:654] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:42.738082 8760 log.cc:824] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 2d38faa555414c018df92fcb0668afa9: Log is configured to *not* fsync() on all Append() calls
I20260808 14:28:42.738173 8759 log.cc:824] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6: Log is configured to *not* fsync() on all Append() calls
I20260808 14:28:42.739445 8761 tablet_bootstrap.cc:654] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 65cbb2a06bde43c8bc0d3d93b26a8e76: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:42.740224 8759 tablet_bootstrap.cc:492] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6: No bootstrap required, opened a new log
I20260808 14:28:42.740314 8760 tablet_bootstrap.cc:492] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 2d38faa555414c018df92fcb0668afa9: No bootstrap required, opened a new log
I20260808 14:28:42.740306 8761 log.cc:824] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 65cbb2a06bde43c8bc0d3d93b26a8e76: Log is configured to *not* fsync() on all Append() calls
I20260808 14:28:42.740439 8759 ts_tablet_manager.cc:1397] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 14:28:42.740494 8760 ts_tablet_manager.cc:1397] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 2d38faa555414c018df92fcb0668afa9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:28:42.742352 8761 tablet_bootstrap.cc:492] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 65cbb2a06bde43c8bc0d3d93b26a8e76: No bootstrap required, opened a new log
I20260808 14:28:42.742525 8761 ts_tablet_manager.cc:1397] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 65cbb2a06bde43c8bc0d3d93b26a8e76: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:28:42.743985 8759 raft_consensus.cc:348] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e425dd69ace248ee807e012e81d47ab6" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 46815 } } peers { permanent_uuid: "2d38faa555414c018df92fcb0668afa9" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44553 } } peers { permanent_uuid: "65cbb2a06bde43c8bc0d3d93b26a8e76" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 32859 } }
I20260808 14:28:42.744238 8759 raft_consensus.cc:374] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:42.744302 8759 raft_consensus.cc:729] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e425dd69ace248ee807e012e81d47ab6, State: Initialized, Role: FOLLOWER
I20260808 14:28:42.744331 8760 raft_consensus.cc:348] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 2d38faa555414c018df92fcb0668afa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e425dd69ace248ee807e012e81d47ab6" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 46815 } } peers { permanent_uuid: "2d38faa555414c018df92fcb0668afa9" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44553 } } peers { permanent_uuid: "65cbb2a06bde43c8bc0d3d93b26a8e76" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 32859 } }
I20260808 14:28:42.744620 8760 raft_consensus.cc:374] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 2d38faa555414c018df92fcb0668afa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:42.744674 8760 raft_consensus.cc:729] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 2d38faa555414c018df92fcb0668afa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d38faa555414c018df92fcb0668afa9, State: Initialized, Role: FOLLOWER
I20260808 14:28:42.744769 8759 consensus_queue.cc:260] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e425dd69ace248ee807e012e81d47ab6" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 46815 } } peers { permanent_uuid: "2d38faa555414c018df92fcb0668afa9" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44553 } } peers { permanent_uuid: "65cbb2a06bde43c8bc0d3d93b26a8e76" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 32859 } }
I20260808 14:28:42.745103 8760 consensus_queue.cc:260] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 2d38faa555414c018df92fcb0668afa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e425dd69ace248ee807e012e81d47ab6" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 46815 } } peers { permanent_uuid: "2d38faa555414c018df92fcb0668afa9" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44553 } } peers { permanent_uuid: "65cbb2a06bde43c8bc0d3d93b26a8e76" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 32859 } }
I20260808 14:28:42.745608 8602 heartbeater.cc:503] Master 127.7.125.126:43777 was elected leader, sending a full tablet report...
I20260808 14:28:42.745836 8759 ts_tablet_manager.cc:1428] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 14:28:42.746479 8760 ts_tablet_manager.cc:1428] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 2d38faa555414c018df92fcb0668afa9: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 14:28:42.746827 8741 heartbeater.cc:503] Master 127.7.125.126:43777 was elected leader, sending a full tablet report...
I20260808 14:28:42.747013 8761 raft_consensus.cc:348] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 65cbb2a06bde43c8bc0d3d93b26a8e76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e425dd69ace248ee807e012e81d47ab6" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 46815 } } peers { permanent_uuid: "2d38faa555414c018df92fcb0668afa9" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44553 } } peers { permanent_uuid: "65cbb2a06bde43c8bc0d3d93b26a8e76" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 32859 } }
I20260808 14:28:42.747308 8761 raft_consensus.cc:374] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 65cbb2a06bde43c8bc0d3d93b26a8e76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:42.747370 8761 raft_consensus.cc:729] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 65cbb2a06bde43c8bc0d3d93b26a8e76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65cbb2a06bde43c8bc0d3d93b26a8e76, State: Initialized, Role: FOLLOWER
I20260808 14:28:42.747815 8761 consensus_queue.cc:260] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 65cbb2a06bde43c8bc0d3d93b26a8e76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e425dd69ace248ee807e012e81d47ab6" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 46815 } } peers { permanent_uuid: "2d38faa555414c018df92fcb0668afa9" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44553 } } peers { permanent_uuid: "65cbb2a06bde43c8bc0d3d93b26a8e76" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 32859 } }
I20260808 14:28:42.748416 8464 heartbeater.cc:503] Master 127.7.125.126:43777 was elected leader, sending a full tablet report...
I20260808 14:28:42.748723 8761 ts_tablet_manager.cc:1428] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 65cbb2a06bde43c8bc0d3d93b26a8e76: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 14:28:42.816398 8765 raft_consensus.cc:482] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:28:42.816648 8765 raft_consensus.cc:504] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e425dd69ace248ee807e012e81d47ab6" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 46815 } } peers { permanent_uuid: "2d38faa555414c018df92fcb0668afa9" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44553 } } peers { permanent_uuid: "65cbb2a06bde43c8bc0d3d93b26a8e76" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 32859 } }
I20260808 14:28:42.817929 8765 leader_election.cc:288] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d38faa555414c018df92fcb0668afa9 (127.7.125.67:44553), 65cbb2a06bde43c8bc0d3d93b26a8e76 (127.7.125.65:32859)
I20260808 14:28:42.822940 8755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:42.818397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:45333 (local address 127.7.125.65:32859)
0808 14:28:42.818536 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:42.818542 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:42.818775 (+ 233us) server_negotiation.cc:407] Connection header received
0808 14:28:42.818881 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:42.818883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:42.818927 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:42.819045 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:28:42.819292 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.820020 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.821394 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.822117 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.822323 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:42.822384 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:42.822723 (+ 339us) server_negotiation.cc:299] Negotiation successful
0808 14:28:42.822809 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 14:28:42.822940 8758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:42.818217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:50337 (local address 127.7.125.67:44553)
0808 14:28:42.818341 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:42.818346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:42.818419 (+ 73us) server_negotiation.cc:407] Connection header received
0808 14:28:42.818580 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:42.818584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:42.818642 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:42.818710 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:28:42.819050 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.819838 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.821394 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.822117 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.822274 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:42.822336 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:42.822723 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 14:28:42.822808 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 14:28:42.823041 8604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:42.818127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:44553 (local address 127.7.125.66:50337)
0808 14:28:42.818326 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:42.818341 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:42.818457 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:42.818760 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:42.818763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:42.818779 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:28:42.818916 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:42.818921 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.819961 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.819963 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:42.821218 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:42.821229 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.822330 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.822335 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:42.822520 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:42.822572 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 14:28:42.822666 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 14:28:42.822852 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":105}
I20260808 14:28:42.823297 8768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:42.818213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:32859 (local address 127.7.125.66:45333)
0808 14:28:42.818642 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:42.818653 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:42.818783 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:42.819056 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:42.819060 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:42.819081 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:28:42.819178 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:42.819183 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.820111 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.820114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:42.821241 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:42.821247 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.822330 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.822335 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:42.822516 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:42.822567 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 14:28:42.822666 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 14:28:42.822852 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":89,"thread_start_us":137,"threads_started":1}
I20260808 14:28:42.823943 8694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b72522f19ea48cd9c0ddb79c4ac3b88" candidate_uuid: "e425dd69ace248ee807e012e81d47ab6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d38faa555414c018df92fcb0668afa9" is_pre_election: true
I20260808 14:28:42.823994 8418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b72522f19ea48cd9c0ddb79c4ac3b88" candidate_uuid: "e425dd69ace248ee807e012e81d47ab6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65cbb2a06bde43c8bc0d3d93b26a8e76" is_pre_election: true
I20260808 14:28:42.824360 8694 raft_consensus.cc:2454] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 2d38faa555414c018df92fcb0668afa9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e425dd69ace248ee807e012e81d47ab6 in term 0.
I20260808 14:28:42.824455 8418 raft_consensus.cc:2454] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 65cbb2a06bde43c8bc0d3d93b26a8e76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e425dd69ace248ee807e012e81d47ab6 in term 0.
I20260808 14:28:42.825007 8488 leader_election.cc:302] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d38faa555414c018df92fcb0668afa9, 65cbb2a06bde43c8bc0d3d93b26a8e76, e425dd69ace248ee807e012e81d47ab6; no voters:
I20260808 14:28:42.825290 8765 raft_consensus.cc:2781] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:28:42.825384 8765 raft_consensus.cc:482] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:28:42.825419 8765 raft_consensus.cc:3037] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:42.826849 8765 raft_consensus.cc:504] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e425dd69ace248ee807e012e81d47ab6" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 46815 } } peers { permanent_uuid: "2d38faa555414c018df92fcb0668afa9" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44553 } } peers { permanent_uuid: "65cbb2a06bde43c8bc0d3d93b26a8e76" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 32859 } }
I20260808 14:28:42.827419 8765 leader_election.cc:288] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6 [CANDIDATE]: Term 1 election: Requested vote from peers 2d38faa555414c018df92fcb0668afa9 (127.7.125.67:44553), 65cbb2a06bde43c8bc0d3d93b26a8e76 (127.7.125.65:32859)
I20260808 14:28:42.827780 8694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b72522f19ea48cd9c0ddb79c4ac3b88" candidate_uuid: "e425dd69ace248ee807e012e81d47ab6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d38faa555414c018df92fcb0668afa9"
I20260808 14:28:42.827791 8418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b72522f19ea48cd9c0ddb79c4ac3b88" candidate_uuid: "e425dd69ace248ee807e012e81d47ab6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65cbb2a06bde43c8bc0d3d93b26a8e76"
I20260808 14:28:42.827919 8694 raft_consensus.cc:3037] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 2d38faa555414c018df92fcb0668afa9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:42.827939 8418 raft_consensus.cc:3037] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 65cbb2a06bde43c8bc0d3d93b26a8e76 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:42.829355 8418 raft_consensus.cc:2454] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 65cbb2a06bde43c8bc0d3d93b26a8e76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e425dd69ace248ee807e012e81d47ab6 in term 1.
I20260808 14:28:42.829355 8694 raft_consensus.cc:2454] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 2d38faa555414c018df92fcb0668afa9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e425dd69ace248ee807e012e81d47ab6 in term 1.
I20260808 14:28:42.829686 8488 leader_election.cc:302] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d38faa555414c018df92fcb0668afa9, e425dd69ace248ee807e012e81d47ab6; no voters:
I20260808 14:28:42.829865 8765 raft_consensus.cc:2781] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:28:42.830120 8765 raft_consensus.cc:686] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6 [term 1 LEADER]: Becoming Leader. State: Replica: e425dd69ace248ee807e012e81d47ab6, State: Running, Role: LEADER
I20260808 14:28:42.830475 8765 consensus_queue.cc:237] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e425dd69ace248ee807e012e81d47ab6" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 46815 } } peers { permanent_uuid: "2d38faa555414c018df92fcb0668afa9" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44553 } } peers { permanent_uuid: "65cbb2a06bde43c8bc0d3d93b26a8e76" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 32859 } }
I20260808 14:28:42.834331 8261 catalog_manager.cc:5495] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6 reported cstate change: term changed from 0 to 1, leader changed from <none> to e425dd69ace248ee807e012e81d47ab6 (127.7.125.66). New cstate: current_term: 1 leader_uuid: "e425dd69ace248ee807e012e81d47ab6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e425dd69ace248ee807e012e81d47ab6" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 46815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d38faa555414c018df92fcb0668afa9" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65cbb2a06bde43c8bc0d3d93b26a8e76" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 32859 } health_report { overall_health: UNKNOWN } } }
Aug 08 14:28:42 dist-test-slave-w206 krb5kdc[8218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.65@KRBTEST.COM
I20260808 14:28:42.875087 8755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:42.865805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47036 (local address 127.7.125.65:32859)
0808 14:28:42.865965 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:42.865969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:42.866169 (+ 200us) server_negotiation.cc:407] Connection header received
0808 14:28:42.866275 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:42.866277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:42.866330 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:42.866400 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:42.867197 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.867726 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.868572 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.868773 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.868855 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:42.871410 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:42.871426 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:42.871442 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:42.871470 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:42.873207 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:42.873652 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:42.873658 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:42.873659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:42.873746 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:42.874080 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:42.874082 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:42.874084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:42.874338 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:42.874515 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:42.874743 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 14:28:42.874870 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 14:28:42 dist-test-slave-w206 krb5kdc[8218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199322, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 14:28:42 dist-test-slave-w206 krb5kdc[8218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199322, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.125.65@KRBTEST.COM
I20260808 14:28:42.904665 8755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:42.895496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47040 (local address 127.7.125.65:32859)
0808 14:28:42.895670 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:42.895676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:42.895818 (+ 142us) server_negotiation.cc:407] Connection header received
0808 14:28:42.896002 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:42.896004 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:42.896049 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:42.896143 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:42.896875 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.897423 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.898343 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.898580 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.898688 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:42.901297 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:42.901301 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:42.901303 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:42.901328 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:42.902797 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:42.903271 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:42.903274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:42.903276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:42.903381 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:42.903724 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:42.903728 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:42.903730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:42.903945 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:42.904077 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:42.904344 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 14:28:42.904513 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260808 14:28:42.912957 8742 tablet.cc:2369] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 2d38faa555414c018df92fcb0668afa9: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 14:28:42 dist-test-slave-w206 krb5kdc[8218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260808 14:28:42.925977 8603 tablet.cc:2369] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P e425dd69ace248ee807e012e81d47ab6: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 14:28:42 dist-test-slave-w206 krb5kdc[8218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.65@KRBTEST.COM
I20260808 14:28:42.932469 8755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:42.923976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47054 (local address 127.7.125.65:32859)
0808 14:28:42.924127 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:42.924132 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:42.924293 (+ 161us) server_negotiation.cc:407] Connection header received
0808 14:28:42.924468 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:42.924471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:42.924513 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:42.924605 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:42.925304 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.925856 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.926824 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:42.927122 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:42.927207 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:42.929700 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:42.929704 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:42.929706 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:42.929729 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:42.931051 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:42.931454 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:42.931461 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:42.931463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:42.931544 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:42.931769 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:42.931771 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:42.931773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:42.931949 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:42.932045 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:42.932239 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 14:28:42.932335 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 14:28:42.934056 8375 tablet_service.cc:2908] Rejecting scan request for tablet 2b72522f19ea48cd9c0ddb79c4ac3b88: Uninitialized: clean time has not yet been initialized
W20260808 14:28:42.934473 8466 tablet.cc:2369] T 2b72522f19ea48cd9c0ddb79c4ac3b88 P 65cbb2a06bde43c8bc0d3d93b26a8e76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:28:42.936745 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 8330
I20260808 14:28:42.946090 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 8470
I20260808 14:28:42.953631 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 8607
I20260808 14:28:42.960163 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 8231
2026-08-08T14:28:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2441 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:28:43 dist-test-slave-w206 krb5kdc[8784](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:28:43 dist-test-slave-w206 krb5kdc[8784](info): set up 2 sockets
Aug 08 14:28:43 dist-test-slave-w206 krb5kdc[8784](info): commencing operation
krb5kdc: starting...
W20260808 14:28:44.043268 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:28:44 dist-test-slave-w206 krb5kdc[8784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:28:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:28:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:28:44.172187 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:35651
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:35747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:35651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:44.287891 8797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:44.288225 8797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:44.288288 8797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:44.292342 8797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:28:44.292446 8797 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 14:28:44.292480 8797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:44.292526 8797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:44.292563 8797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:28:44.292600 8797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:28:44.298036 8797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:35747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:35651
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:35651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:44.299228 8797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:44.300343 8797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:28:44.309322 8802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:44.309337 8805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:44.309337 8803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:44.309682 8797 server_base.cc:1034] running on GCE node
I20260808 14:28:44.310216 8797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:44.311394 8797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:44.312563 8797 hybrid_clock.cc:648] HybridClock initialized: now 1786199324312545 us; error 31 us; skew 500 ppm
Aug 08 14:28:44 dist-test-slave-w206 krb5kdc[8784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199324, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:44.315551 8797 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:28:44.316543 8797 webserver.cc:460] Webserver started at http://127.7.125.126:46539/ using document root <none> and password file <none>
I20260808 14:28:44.317119 8797 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:44.317214 8797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:44.317461 8797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:44.319644 8797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "631f45d24031409eafa1d5f91eeda47b"
format_stamp: "Formatted at 2026-08-08 14:28:44 on dist-test-slave-w206"
server_key: "52f4bdbddf38f41425189509d39f12a4"
server_key_iv: "072a987f106af3a78926217fd2751b95"
server_key_version: "encryptionkey@0"
I20260808 14:28:44.320257 8797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "631f45d24031409eafa1d5f91eeda47b"
format_stamp: "Formatted at 2026-08-08 14:28:44 on dist-test-slave-w206"
server_key: "52f4bdbddf38f41425189509d39f12a4"
server_key_iv: "072a987f106af3a78926217fd2751b95"
server_key_version: "encryptionkey@0"
I20260808 14:28:44.324049 8797 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 14:28:44.326462 8812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:44.327781 8797 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:28:44.327929 8797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "631f45d24031409eafa1d5f91eeda47b"
format_stamp: "Formatted at 2026-08-08 14:28:44 on dist-test-slave-w206"
server_key: "52f4bdbddf38f41425189509d39f12a4"
server_key_iv: "072a987f106af3a78926217fd2751b95"
server_key_version: "encryptionkey@0"
I20260808 14:28:44.328045 8797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:44.359086 8797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:44.362707 8797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:44.362947 8797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:44.371675 8797 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:35651
I20260808 14:28:44.371697 8875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:35651 every 8 connection(s)
I20260808 14:28:44.372632 8797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:28:44.375429 8876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:44.379962 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 8797
I20260808 14:28:44.380090 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:28:44.380362 7669 external_mini_cluster.cc:1428] Setting key 78de9797f512de3e0f32bf23f9b5388e
I20260808 14:28:44.381534 8876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b: Bootstrap starting.
I20260808 14:28:44.383947 8876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:44.384678 8876 log.cc:824] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b: Log is configured to *not* fsync() on all Append() calls
I20260808 14:28:44.386929 8876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b: No bootstrap required, opened a new log
Aug 08 14:28:44 dist-test-slave-w206 krb5kdc[8784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:44.391004 8876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631f45d24031409eafa1d5f91eeda47b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35651 } }
I20260808 14:28:44.391322 8876 raft_consensus.cc:374] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:44.391387 8876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 631f45d24031409eafa1d5f91eeda47b, State: Initialized, Role: FOLLOWER
I20260808 14:28:44.391850 8876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631f45d24031409eafa1d5f91eeda47b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35651 } }
I20260808 14:28:44.391987 8876 raft_consensus.cc:388] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:28:44.392082 8876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:28:44.392189 8876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:44.393249 8876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631f45d24031409eafa1d5f91eeda47b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35651 } }
I20260808 14:28:44.393590 8876 leader_election.cc:302] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 631f45d24031409eafa1d5f91eeda47b; no voters:
I20260808 14:28:44.393972 8876 leader_election.cc:288] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:28:44.394119 8882 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:28:44.394367 8882 raft_consensus.cc:686] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b [term 1 LEADER]: Becoming Leader. State: Replica: 631f45d24031409eafa1d5f91eeda47b, State: Running, Role: LEADER
I20260808 14:28:44.394699 8882 consensus_queue.cc:237] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631f45d24031409eafa1d5f91eeda47b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35651 } }
I20260808 14:28:44.395314 8876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:28:44.396474 8883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "631f45d24031409eafa1d5f91eeda47b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631f45d24031409eafa1d5f91eeda47b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35651 } } }
I20260808 14:28:44.396615 8883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b [sys.catalog]: This master's current role is: LEADER
I20260808 14:28:44.396987 8884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 631f45d24031409eafa1d5f91eeda47b. Latest consensus state: current_term: 1 leader_uuid: "631f45d24031409eafa1d5f91eeda47b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631f45d24031409eafa1d5f91eeda47b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35651 } } }
I20260808 14:28:44.397083 8884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b [sys.catalog]: This master's current role is: LEADER
I20260808 14:28:44.397048 8891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:28:44.397413 8880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:44.381979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38956 (local address 127.7.125.126:35651)
0808 14:28:44.382458 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:44.382475 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:44.382516 (+ 41us) server_negotiation.cc:407] Connection header received
0808 14:28:44.383402 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:44.383422 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:44.383849 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:44.384292 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:44.385299 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:44.386716 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:44.387760 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:44.388551 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:44.388710 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:44.391704 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:44.391710 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:44.391727 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:44.391765 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:44.393562 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:44.394138 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:44.394146 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:44.394152 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:44.394307 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:44.394643 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:44.394646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:44.394648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:44.395190 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:44.395348 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:44.395944 (+ 596us) server_negotiation.cc:299] Negotiation successful
0808 14:28:44.396283 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":138,"threads_started":1}
I20260808 14:28:44.400832 8891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:28:44.410681 8891 catalog_manager.cc:1347] Generated new cluster ID: 91385d0ba29e416f896503cb1a1b9d5e
I20260808 14:28:44.410759 8891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:28:44.422755 8891 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:28:44.424049 8891 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:28:44.428999 8891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b: Generated new TSK 0
I20260808 14:28:44.429692 8891 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.125.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
I20260808 14:28:44.458455 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:35651
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:35747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:44.577831 8903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:44.578119 8903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:44.578171 8903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:44.582872 8903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:44.582952 8903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:44.583014 8903 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 14:28:44.583055 8903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:28:44.589320 8903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:35747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.125.126:35651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:44.590471 8903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:44.591579 8903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:28:44.602676 8912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:44.602696 8908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:44.602701 8909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:44.603264 8903 server_base.cc:1034] running on GCE node
I20260808 14:28:44.603664 8903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:44.604374 8903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:44.605561 8903 hybrid_clock.cc:648] HybridClock initialized: now 1786199324605536 us; error 48 us; skew 500 ppm
Aug 08 14:28:44 dist-test-slave-w206 krb5kdc[8784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199324, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:44.608500 8903 init.cc:377] Logged in from keytab as kudu/127.7.125.65@KRBTEST.COM (short username kudu)
I20260808 14:28:44.609385 8903 webserver.cc:460] Webserver started at http://127.7.125.65:37985/ using document root <none> and password file <none>
I20260808 14:28:44.609977 8903 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:44.610066 8903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:44.610309 8903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:44.612656 8903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "b45118b88b48485fa2a5a5814e564c36"
format_stamp: "Formatted at 2026-08-08 14:28:44 on dist-test-slave-w206"
server_key: "f5ed4c54b7b9e6b907a462655d8d8041"
server_key_iv: "791dd1802d6e90526b00001cf724b63a"
server_key_version: "encryptionkey@0"
I20260808 14:28:44.613183 8903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "b45118b88b48485fa2a5a5814e564c36"
format_stamp: "Formatted at 2026-08-08 14:28:44 on dist-test-slave-w206"
server_key: "f5ed4c54b7b9e6b907a462655d8d8041"
server_key_iv: "791dd1802d6e90526b00001cf724b63a"
server_key_version: "encryptionkey@0"
I20260808 14:28:44.616858 8903 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 14:28:44.619234 8919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:44.620465 8903 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:28:44.620558 8903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "b45118b88b48485fa2a5a5814e564c36"
format_stamp: "Formatted at 2026-08-08 14:28:44 on dist-test-slave-w206"
server_key: "f5ed4c54b7b9e6b907a462655d8d8041"
server_key_iv: "791dd1802d6e90526b00001cf724b63a"
server_key_version: "encryptionkey@0"
I20260808 14:28:44.620642 8903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:44.657823 8903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:44.660770 8903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:44.660943 8903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:44.662344 8903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:28:44.662431 8903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:44.662493 8903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:28:44.662541 8903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 14:28:44 dist-test-slave-w206 krb5kdc[8784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199324, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:44.673934 8903 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:34545
I20260808 14:28:44.674021 9039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:34545 every 8 connection(s)
I20260808 14:28:44.675088 8903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:28:44.675859 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 8903
I20260808 14:28:44.675971 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:28:44.676224 7669 external_mini_cluster.cc:1428] Setting key dfc7667e9d93cc932d8e484f77a7aa6b
I20260808 14:28:44.678102 8880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:44.663325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:54693 (local address 127.7.125.126:35651)
0808 14:28:44.663452 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:44.663455 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:44.664546 (+ 1091us) server_negotiation.cc:407] Connection header received
0808 14:28:44.665648 (+ 1102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:44.665654 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:44.665721 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:44.665790 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:44.667613 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:44.668406 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:44.669583 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:44.670014 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:44.670147 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:44.672921 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:44.672925 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:44.672927 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:44.672964 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:44.674736 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:44.675375 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:44.675381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:44.675384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:44.675490 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:44.676911 (+ 1421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:44.676914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:44.676916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:44.677164 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:44.677265 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:44.677863 (+ 598us) server_negotiation.cc:299] Negotiation successful
0808 14:28:44.677983 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 14:28:44.679211 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:44.663900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35651 (local address 127.7.125.65:54693)
0808 14:28:44.664396 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:44.664440 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:44.665387 (+ 947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:44.666267 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:44.666276 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:44.666753 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:44.667437 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:44.667459 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:44.668543 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:44.668547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:44.669435 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:44.669445 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:44.670207 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:44.670215 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:44.670467 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:44.671198 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:44.671215 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:44.672729 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:44.674850 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:44.674854 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:44.674865 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:44.675273 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:44.676603 (+ 1330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:44.676606 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:44.676607 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:44.676783 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:44.677271 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:44.677276 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:44.677584 (+ 308us) client_negotiation.cc:769] Sending connection context
0808 14:28:44.677790 (+ 206us) client_negotiation.cc:240] Negotiation successful
0808 14:28:44.678120 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":143,"threads_started":1}
I20260808 14:28:44.680516 9040 heartbeater.cc:347] Connected to a master server at 127.7.125.126:35651
I20260808 14:28:44.680655 9040 heartbeater.cc:464] Registering TS with master...
I20260808 14:28:44.681211 9040 heartbeater.cc:511] Master 127.7.125.126:35651 requested a full tablet report, sending...
I20260808 14:28:44.682787 8828 ts_manager.cc:194] Registered new tserver with Master: b45118b88b48485fa2a5a5814e564c36 (127.7.125.65:34545)
I20260808 14:28:44.684319 8828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:54693
WARNING: no policy specified for kudu/127.7.125.66@KRBTEST.COM; defaulting to no policy
I20260808 14:28:44.691097 8880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:44.683033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:45897 (local address 127.7.125.126:35651)
0808 14:28:44.683167 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:44.683174 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:44.683383 (+ 209us) server_negotiation.cc:407] Connection header received
0808 14:28:44.683480 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:44.683486 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:44.683540 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:44.683626 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:44.684130 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:44.685003 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:44.686146 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:44.686364 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:44.686448 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:44.687889 (+ 1441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:44.687892 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:44.687894 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:44.687915 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:44.689342 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:44.689838 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:44.689840 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:44.689841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:44.689912 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:44.690225 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:44.690227 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:44.690228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:44.690385 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:44.690459 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:44.690786 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 14:28:44.690890 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":44}
I20260808 14:28:44.691237 9048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:44.682934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35651 (local address 127.7.125.65:45897)
0808 14:28:44.683291 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:44.683306 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:44.683402 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:44.683664 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:44.683667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:44.683899 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:44.684012 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:44.684019 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:44.685170 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:44.685173 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:44.686024 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:44.686033 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:44.686483 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:44.686487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:44.686579 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:44.687219 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:44.687239 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:44.687749 (+ 510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:44.689460 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:44.689464 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:44.689466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:44.689727 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:44.690007 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:44.690014 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:44.690018 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:44.690128 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:44.690491 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:44.690498 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:44.690609 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 14:28:44.690729 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 14:28:44.690873 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":134,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
I20260808 14:28:44.710386 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:35651
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:35747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:44.826730 9050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:44.827064 9050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:44.827181 9050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:44.831086 9050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:44.831171 9050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:44.831244 9050 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 14:28:44.831326 9050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:28:44.836546 9050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:35747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.125.126:35651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:44.837625 9050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:44.838666 9050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:28:44.848304 9055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:44.848323 9056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:44.848369 9050 server_base.cc:1034] running on GCE node
W20260808 14:28:44.848320 9058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:44.848999 9050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:44.849704 9050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:44.850868 9050 hybrid_clock.cc:648] HybridClock initialized: now 1786199324850854 us; error 25 us; skew 500 ppm
Aug 08 14:28:44 dist-test-slave-w206 krb5kdc[8784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199324, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:44.853705 9050 init.cc:377] Logged in from keytab as kudu/127.7.125.66@KRBTEST.COM (short username kudu)
I20260808 14:28:44.854569 9050 webserver.cc:460] Webserver started at http://127.7.125.66:41359/ using document root <none> and password file <none>
I20260808 14:28:44.855180 9050 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:44.855242 9050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:44.855497 9050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:44.857614 9050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "c7486dac41a34864b4f2327f47fc24ad"
format_stamp: "Formatted at 2026-08-08 14:28:44 on dist-test-slave-w206"
server_key: "88fc8a18cc79fdc33b4c70a3604cd4b5"
server_key_iv: "7055a277e23e8285e6efc8d0ed417bba"
server_key_version: "encryptionkey@0"
I20260808 14:28:44.858204 9050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "c7486dac41a34864b4f2327f47fc24ad"
format_stamp: "Formatted at 2026-08-08 14:28:44 on dist-test-slave-w206"
server_key: "88fc8a18cc79fdc33b4c70a3604cd4b5"
server_key_iv: "7055a277e23e8285e6efc8d0ed417bba"
server_key_version: "encryptionkey@0"
I20260808 14:28:44.861891 9050 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 14:28:44.864316 9066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:44.865515 9050 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:28:44.865649 9050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "c7486dac41a34864b4f2327f47fc24ad"
format_stamp: "Formatted at 2026-08-08 14:28:44 on dist-test-slave-w206"
server_key: "88fc8a18cc79fdc33b4c70a3604cd4b5"
server_key_iv: "7055a277e23e8285e6efc8d0ed417bba"
server_key_version: "encryptionkey@0"
I20260808 14:28:44.865759 9050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:44.884732 9050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:44.887975 9050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:44.888187 9050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:44.889715 9050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:28:44.889801 9050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:44.889873 9050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:28:44.889919 9050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 14:28:44 dist-test-slave-w206 krb5kdc[8784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199324, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:44.900782 9050 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:45975
I20260808 14:28:44.901300 9181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:45975 every 8 connection(s)
I20260808 14:28:44.902205 9050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:28:44.904630 8880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:44.890633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:45977 (local address 127.7.125.126:35651)
0808 14:28:44.890763 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:44.890766 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:44.891885 (+ 1119us) server_negotiation.cc:407] Connection header received
0808 14:28:44.892940 (+ 1055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:44.892943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:44.893005 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:44.893091 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:44.894781 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:44.895676 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:44.897523 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:44.897881 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:44.898009 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:44.900845 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:44.900848 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:44.900851 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:44.900876 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:44.902631 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:44.903292 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:44.903295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:44.903296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:44.903378 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:44.903701 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:44.903703 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:44.903704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:44.903873 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:44.903947 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:44.904416 (+ 469us) server_negotiation.cc:299] Negotiation successful
0808 14:28:44.904513 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 14:28:44.905398 9075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:44.891235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35651 (local address 127.7.125.66:45977)
0808 14:28:44.891733 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:44.891776 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:44.892721 (+ 945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:44.893336 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:44.893349 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:44.893867 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:44.894593 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:44.894619 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:44.895868 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:44.895874 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:44.897348 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:44.897362 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:44.898047 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:44.898051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:44.898186 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:44.898902 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:44.898920 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:44.900653 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:44.902752 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:44.902761 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:44.902781 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:44.903162 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:44.903471 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:44.903474 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:44.903475 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:44.903608 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:44.903962 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:44.903967 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:44.904172 (+ 205us) client_negotiation.cc:769] Sending connection context
0808 14:28:44.904317 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 14:28:44.904557 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":142,"threads_started":1}
I20260808 14:28:44.906661 9182 heartbeater.cc:347] Connected to a master server at 127.7.125.126:35651
I20260808 14:28:44.906795 9182 heartbeater.cc:464] Registering TS with master...
I20260808 14:28:44.907373 9182 heartbeater.cc:511] Master 127.7.125.126:35651 requested a full tablet report, sending...
I20260808 14:28:44.907523 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 9050
I20260808 14:28:44.907660 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:28:44.907958 7669 external_mini_cluster.cc:1428] Setting key a2d6a032e653d7e911665a894a66fe9f
I20260808 14:28:44.908564 8829 ts_manager.cc:194] Registered new tserver with Master: c7486dac41a34864b4f2327f47fc24ad (127.7.125.66:45975)
I20260808 14:28:44.909215 8829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:45977
I20260808 14:28:44.916548 8880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:44.909233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:45137 (local address 127.7.125.126:35651)
0808 14:28:44.909660 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:44.909664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:44.909678 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:28:44.909712 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:44.909715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:44.909753 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:44.909837 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:44.910267 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:44.910772 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:44.911681 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:44.911901 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:44.912024 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:44.913285 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:44.913288 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:44.913290 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:44.913333 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:44.914768 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:44.915302 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:44.915305 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:44.915306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:44.915393 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:44.915707 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:44.915709 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:44.915711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:44.915908 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:44.915985 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:44.916340 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 14:28:44.916434 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337}
I20260808 14:28:44.916703 9188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:44.909110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35651 (local address 127.7.125.66:45137)
0808 14:28:44.909463 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:44.909481 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:44.909574 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:44.909869 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:44.909872 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:44.910053 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:44.910147 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:44.910152 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:44.910911 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:44.910914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:44.911553 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:44.911562 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:44.912029 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:44.912033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:44.912126 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:44.912661 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:44.912681 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:44.913156 (+ 475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:44.914898 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:44.914904 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:44.914908 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:44.915203 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:44.915491 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:44.915497 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:44.915502 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:44.915605 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:44.916010 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:44.916016 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:44.916136 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 14:28:44.916261 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 14:28:44.916432 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":2,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for kudu/127.7.125.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
I20260808 14:28:44.938956 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:35651
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:35747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:45.051486 9191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:45.051779 9191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:45.051853 9191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:45.056020 9191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:45.056107 9191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:45.056180 9191 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 14:28:45.056260 9191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:28:45.061647 9191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:35747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.125.126:35651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:45.062741 9191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:45.063864 9191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:28:45.073720 9198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:45.073714 9200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:45.073714 9197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:45.074234 9191 server_base.cc:1034] running on GCE node
I20260808 14:28:45.074620 9191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:45.075330 9191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:45.076524 9191 hybrid_clock.cc:648] HybridClock initialized: now 1786199325076509 us; error 34 us; skew 500 ppm
Aug 08 14:28:45 dist-test-slave-w206 krb5kdc[8784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199325, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:45.079216 9191 init.cc:377] Logged in from keytab as kudu/127.7.125.67@KRBTEST.COM (short username kudu)
I20260808 14:28:45.080093 9191 webserver.cc:460] Webserver started at http://127.7.125.67:33743/ using document root <none> and password file <none>
I20260808 14:28:45.080679 9191 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:45.080770 9191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:45.080999 9191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:45.083549 9191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "f017f17924f2495e92d334bf24a622d1"
format_stamp: "Formatted at 2026-08-08 14:28:45 on dist-test-slave-w206"
server_key: "3bca93f759deb4477ccc7c105ed44993"
server_key_iv: "0253d2ff3da30684ba3c28e97cfb69c5"
server_key_version: "encryptionkey@0"
I20260808 14:28:45.084116 9191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "f017f17924f2495e92d334bf24a622d1"
format_stamp: "Formatted at 2026-08-08 14:28:45 on dist-test-slave-w206"
server_key: "3bca93f759deb4477ccc7c105ed44993"
server_key_iv: "0253d2ff3da30684ba3c28e97cfb69c5"
server_key_version: "encryptionkey@0"
I20260808 14:28:45.087968 9191 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 14:28:45.090346 9207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:45.091624 9191 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:28:45.091758 9191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "f017f17924f2495e92d334bf24a622d1"
format_stamp: "Formatted at 2026-08-08 14:28:45 on dist-test-slave-w206"
server_key: "3bca93f759deb4477ccc7c105ed44993"
server_key_iv: "0253d2ff3da30684ba3c28e97cfb69c5"
server_key_version: "encryptionkey@0"
I20260808 14:28:45.091867 9191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:45.118100 9191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:45.121191 9191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:45.121420 9191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:45.123018 9191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:28:45.123102 9191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:45.123172 9191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:28:45.123237 9191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:45.134418 9191 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:40843
Aug 08 14:28:45 dist-test-slave-w206 krb5kdc[8784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199325, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:45.134487 9325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:40843 every 8 connection(s)
I20260808 14:28:45.135432 9191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:28:45.136320 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 9191
I20260808 14:28:45.136447 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:28:45.136710 7669 external_mini_cluster.cc:1428] Setting key 11e0b9dd73f49e6d56e6563a74fe63b9
I20260808 14:28:45.139302 8880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.123817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:35011 (local address 127.7.125.126:35651)
0808 14:28:45.123996 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:45.124001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:45.125647 (+ 1646us) server_negotiation.cc:407] Connection header received
0808 14:28:45.127096 (+ 1449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:45.127100 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:45.127166 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:45.127270 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:45.129083 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.129726 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.131705 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.131924 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.132006 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.135140 (+ 3134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:45.135144 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:45.135146 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:45.135182 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:45.136836 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:45.137454 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:45.137458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:45.137459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:45.137557 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:45.137983 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:45.137988 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:45.137990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:45.138252 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:45.138355 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:45.139067 (+ 712us) server_negotiation.cc:299] Negotiation successful
0808 14:28:45.139185 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 14:28:45.140185 9217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.124870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35651 (local address 127.7.125.67:35011)
0808 14:28:45.125467 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:45.125533 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:45.126823 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:45.127491 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:45.127501 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:45.128074 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:45.128864 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.128888 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.129884 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.129888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.131568 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.131583 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.132104 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.132111 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.132351 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.133094 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:45.133122 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:45.134938 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:45.136964 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:45.136972 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:45.136989 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:45.137335 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:45.137655 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:45.137661 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:45.137664 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:45.137867 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:45.138382 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:45.138387 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:45.138728 (+ 341us) client_negotiation.cc:769] Sending connection context
0808 14:28:45.138945 (+ 217us) client_negotiation.cc:240] Negotiation successful
0808 14:28:45.139257 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":162,"threads_started":1}
I20260808 14:28:45.141420 9326 heartbeater.cc:347] Connected to a master server at 127.7.125.126:35651
I20260808 14:28:45.141573 9326 heartbeater.cc:464] Registering TS with master...
I20260808 14:28:45.142062 9326 heartbeater.cc:511] Master 127.7.125.126:35651 requested a full tablet report, sending...
I20260808 14:28:45.143108 8828 ts_manager.cc:194] Registered new tserver with Master: f017f17924f2495e92d334bf24a622d1 (127.7.125.67:40843)
I20260808 14:28:45.143784 8828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:35011
I20260808 14:28:45.150712 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.143726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35651 (local address 127.7.125.67:55675)
0808 14:28:45.144013 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:45.144030 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:45.144165 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:45.144434 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:45.144437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:45.144662 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:45.144775 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.144783 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.145521 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.145524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.146171 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.146180 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.146597 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.146599 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.146665 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.147199 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:45.147210 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:45.147523 (+ 313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:45.148970 (+ 1447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:45.148996 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:45.149000 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:45.149245 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:45.149577 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:45.149583 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:45.149586 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:45.149714 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:45.150142 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:45.150145 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:45.150259 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 14:28:45.150385 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 14:28:45.150498 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":113,"threads_started":1}
I20260808 14:28:45.150743 8880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.143797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:55675 (local address 127.7.125.126:35651)
0808 14:28:45.143941 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:45.143947 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:45.144097 (+ 150us) server_negotiation.cc:407] Connection header received
0808 14:28:45.144260 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:45.144263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:45.144313 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:45.144415 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:45.144900 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.145388 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.146279 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.146501 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.146579 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.147643 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:45.147646 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:45.147648 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:45.147669 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:45.148874 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:45.149356 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:45.149359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:45.149361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:45.149474 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:45.149820 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:45.149823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:45.149825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:45.150022 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:45.150148 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:45.150469 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 14:28:45.150594 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 14:28:45.150826 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 14:28:45 dist-test-slave-w206 krb5kdc[8784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199325, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 14:28:45 dist-test-slave-w206 krb5kdc[8784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199325, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:45.179092 8880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.170648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38962 (local address 127.7.125.126:35651)
0808 14:28:45.170797 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:45.170801 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:45.170822 (+ 21us) server_negotiation.cc:407] Connection header received
0808 14:28:45.170864 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:45.170866 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:45.170905 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:45.171035 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:45.171781 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.172316 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.173154 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.173380 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.173490 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.176056 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:45.176060 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:45.176061 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:45.176087 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:45.177333 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:45.177785 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:45.177789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:45.177790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:45.177865 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:45.178150 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:45.178153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:45.178155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:45.178345 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:45.178489 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:45.178766 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 14:28:45.178893 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 14:28:45.182332 8828 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:28:45.185333 8828 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:28:45.200343 9348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.193587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:45975 (local address 127.0.0.1:44384)
0808 14:28:45.195219 (+ 1632us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:45.195231 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:45.195341 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:45.195815 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:45.195818 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:45.195839 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:28:45.195943 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.195949 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.197554 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.197557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.198638 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.198649 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.199695 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.199698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.199905 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.199941 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 14:28:45.200024 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 14:28:45.200169 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1565,"mutex_wait_us":57,"thread_start_us":99,"threads_started":1}
I20260808 14:28:45.200341 9347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.193722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59988 (local address 127.7.125.67:40843)
0808 14:28:45.194442 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:45.194456 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:45.194482 (+ 26us) server_negotiation.cc:407] Connection header received
0808 14:28:45.194560 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:45.194567 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:45.194786 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:45.195435 (+ 649us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:28:45.195579 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.196868 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.198671 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.199538 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.199663 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.199859 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:45.200107 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 14:28:45.200194 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":125,"threads_started":1}
I20260808 14:28:45.200363 9349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.194715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44384 (local address 127.7.125.66:45975)
0808 14:28:45.195120 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:45.195133 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:45.195343 (+ 210us) server_negotiation.cc:407] Connection header received
0808 14:28:45.195479 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:45.195484 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:45.195673 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:45.195822 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:28:45.196059 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.197421 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.198764 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.199550 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.199839 (+ 289us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.200035 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:45.200091 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 14:28:45.200195 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":100,"threads_started":1}
I20260808 14:28:45.200826 9346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.193577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:40843 (local address 127.0.0.1:59988)
0808 14:28:45.193969 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:45.194017 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:45.194138 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:45.195104 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:45.195112 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:45.195140 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:28:45.195309 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.195318 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.196999 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.197002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.198543 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.198554 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.199672 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.199677 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.199884 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.199939 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 14:28:45.200599 (+ 660us) client_negotiation.cc:240] Negotiation successful
0808 14:28:45.200684 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":104,"thread_start_us":116,"threads_started":1}
I20260808 14:28:45.201210 9345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.194308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:34545 (local address 127.0.0.1:56038)
0808 14:28:45.195537 (+ 1229us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:45.195547 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:45.195602 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:45.196298 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:45.196302 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:45.196319 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:28:45.196453 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.196510 (+ 57us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.197835 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.197840 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.198754 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.198762 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.199747 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.199751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.200925 (+ 1174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.200947 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 14:28:45.201012 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 14:28:45.201078 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1168,"mutex_wait_us":1034,"thread_start_us":101,"threads_started":1}
I20260808 14:28:45.201800 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.194852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56038 (local address 127.7.125.65:34545)
0808 14:28:45.195827 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:45.195833 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:45.195854 (+ 21us) server_negotiation.cc:407] Connection header received
0808 14:28:45.195926 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:45.195931 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:45.196101 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:45.196236 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:28:45.196662 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.197724 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.198852 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.199616 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.201375 (+ 1759us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.201519 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:45.201575 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 14:28:45.201673 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":854,"thread_start_us":86,"threads_started":1}
I20260808 14:28:45.203617 9116 tablet_service.cc:1467] Processing CreateTablet for tablet 51458d32cb0e45419694058d98a5260b (DEFAULT_TABLE table=test-table [id=c3961ffcc80f41e48973f53d4c338efd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:28:45.203846 9259 tablet_service.cc:1467] Processing CreateTablet for tablet 51458d32cb0e45419694058d98a5260b (DEFAULT_TABLE table=test-table [id=c3961ffcc80f41e48973f53d4c338efd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:28:45.204811 9116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51458d32cb0e45419694058d98a5260b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:45.204478 8972 tablet_service.cc:1467] Processing CreateTablet for tablet 51458d32cb0e45419694058d98a5260b (DEFAULT_TABLE table=test-table [id=c3961ffcc80f41e48973f53d4c338efd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:28:45.204897 9259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51458d32cb0e45419694058d98a5260b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:45.205358 8972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51458d32cb0e45419694058d98a5260b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:45.211968 9351 tablet_bootstrap.cc:492] T 51458d32cb0e45419694058d98a5260b P f017f17924f2495e92d334bf24a622d1: Bootstrap starting.
I20260808 14:28:45.213080 9352 tablet_bootstrap.cc:492] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36: Bootstrap starting.
I20260808 14:28:45.213375 9353 tablet_bootstrap.cc:492] T 51458d32cb0e45419694058d98a5260b P c7486dac41a34864b4f2327f47fc24ad: Bootstrap starting.
I20260808 14:28:45.214212 9351 tablet_bootstrap.cc:654] T 51458d32cb0e45419694058d98a5260b P f017f17924f2495e92d334bf24a622d1: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:45.215224 9351 log.cc:824] T 51458d32cb0e45419694058d98a5260b P f017f17924f2495e92d334bf24a622d1: Log is configured to *not* fsync() on all Append() calls
I20260808 14:28:45.215258 9353 tablet_bootstrap.cc:654] T 51458d32cb0e45419694058d98a5260b P c7486dac41a34864b4f2327f47fc24ad: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:45.215258 9352 tablet_bootstrap.cc:654] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:45.216086 9352 log.cc:824] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36: Log is configured to *not* fsync() on all Append() calls
I20260808 14:28:45.216086 9353 log.cc:824] T 51458d32cb0e45419694058d98a5260b P c7486dac41a34864b4f2327f47fc24ad: Log is configured to *not* fsync() on all Append() calls
I20260808 14:28:45.217298 9351 tablet_bootstrap.cc:492] T 51458d32cb0e45419694058d98a5260b P f017f17924f2495e92d334bf24a622d1: No bootstrap required, opened a new log
I20260808 14:28:45.217514 9351 ts_tablet_manager.cc:1397] T 51458d32cb0e45419694058d98a5260b P f017f17924f2495e92d334bf24a622d1: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 14:28:45.217832 9353 tablet_bootstrap.cc:492] T 51458d32cb0e45419694058d98a5260b P c7486dac41a34864b4f2327f47fc24ad: No bootstrap required, opened a new log
I20260808 14:28:45.218041 9353 ts_tablet_manager.cc:1397] T 51458d32cb0e45419694058d98a5260b P c7486dac41a34864b4f2327f47fc24ad: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 14:28:45.220325 9352 tablet_bootstrap.cc:492] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36: No bootstrap required, opened a new log
I20260808 14:28:45.220526 9352 ts_tablet_manager.cc:1397] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260808 14:28:45.220672 9351 raft_consensus.cc:348] T 51458d32cb0e45419694058d98a5260b P f017f17924f2495e92d334bf24a622d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } }
I20260808 14:28:45.220940 9351 raft_consensus.cc:374] T 51458d32cb0e45419694058d98a5260b P f017f17924f2495e92d334bf24a622d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:45.221014 9351 raft_consensus.cc:729] T 51458d32cb0e45419694058d98a5260b P f017f17924f2495e92d334bf24a622d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f017f17924f2495e92d334bf24a622d1, State: Initialized, Role: FOLLOWER
I20260808 14:28:45.221493 9351 consensus_queue.cc:260] T 51458d32cb0e45419694058d98a5260b P f017f17924f2495e92d334bf24a622d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } }
I20260808 14:28:45.221724 9353 raft_consensus.cc:348] T 51458d32cb0e45419694058d98a5260b P c7486dac41a34864b4f2327f47fc24ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } }
I20260808 14:28:45.222060 9353 raft_consensus.cc:374] T 51458d32cb0e45419694058d98a5260b P c7486dac41a34864b4f2327f47fc24ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:45.222143 9353 raft_consensus.cc:729] T 51458d32cb0e45419694058d98a5260b P c7486dac41a34864b4f2327f47fc24ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7486dac41a34864b4f2327f47fc24ad, State: Initialized, Role: FOLLOWER
I20260808 14:28:45.222568 9351 ts_tablet_manager.cc:1428] T 51458d32cb0e45419694058d98a5260b P f017f17924f2495e92d334bf24a622d1: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260808 14:28:45.222602 9353 consensus_queue.cc:260] T 51458d32cb0e45419694058d98a5260b P c7486dac41a34864b4f2327f47fc24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } }
I20260808 14:28:45.222796 9326 heartbeater.cc:503] Master 127.7.125.126:35651 was elected leader, sending a full tablet report...
I20260808 14:28:45.223387 9182 heartbeater.cc:503] Master 127.7.125.126:35651 was elected leader, sending a full tablet report...
I20260808 14:28:45.223717 9353 ts_tablet_manager.cc:1428] T 51458d32cb0e45419694058d98a5260b P c7486dac41a34864b4f2327f47fc24ad: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 14:28:45.225193 9352 raft_consensus.cc:348] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } }
I20260808 14:28:45.225404 9352 raft_consensus.cc:374] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:45.225476 9352 raft_consensus.cc:729] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b45118b88b48485fa2a5a5814e564c36, State: Initialized, Role: FOLLOWER
I20260808 14:28:45.225867 9352 consensus_queue.cc:260] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } }
I20260808 14:28:45.226540 9040 heartbeater.cc:503] Master 127.7.125.126:35651 was elected leader, sending a full tablet report...
I20260808 14:28:45.227533 9352 ts_tablet_manager.cc:1428] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 14:28:45.341596 9359 raft_consensus.cc:482] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:28:45.341939 9359 raft_consensus.cc:504] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } }
I20260808 14:28:45.343395 9359 leader_election.cc:288] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f017f17924f2495e92d334bf24a622d1 (127.7.125.67:40843), c7486dac41a34864b4f2327f47fc24ad (127.7.125.66:45975)
I20260808 14:28:45.347373 9349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.343917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:35747 (local address 127.7.125.66:45975)
0808 14:28:45.344044 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:45.344049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:45.344172 (+ 123us) server_negotiation.cc:407] Connection header received
0808 14:28:45.344328 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:45.344336 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:45.344377 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:45.344466 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:28:45.344739 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.345245 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.346366 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.346778 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.346919 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.346997 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:45.347167 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 14:28:45.347235 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 14:28:45.347399 9347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.343720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:56683 (local address 127.7.125.67:40843)
0808 14:28:45.343865 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:45.343870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:45.344100 (+ 230us) server_negotiation.cc:407] Connection header received
0808 14:28:45.344262 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:45.344265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:45.344322 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:45.344408 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:28:45.344689 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.345219 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.346375 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.346794 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.346928 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.346981 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:45.347196 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 14:28:45.347264 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 14:28:45.347427 9361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.343617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:40843 (local address 127.7.125.65:56683)
0808 14:28:45.344018 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:45.344032 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:45.344147 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:45.344421 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:45.344424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:45.344439 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:28:45.344578 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.344583 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.345343 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.345349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.346243 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.346252 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.346919 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.346923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.347061 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.347085 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 14:28:45.347177 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 14:28:45.347253 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":23,"thread_start_us":136,"threads_started":1}
I20260808 14:28:45.347445 9360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.343674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:45975 (local address 127.7.125.65:35747)
0808 14:28:45.344065 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:45.344077 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:45.344205 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:45.344480 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:45.344483 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:45.344500 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:28:45.344622 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.344628 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.345363 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.345368 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.346234 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.346244 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.346908 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.346912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.347033 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.347062 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 14:28:45.347149 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 14:28:45.347229 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":28,"thread_start_us":135,"threads_started":1}
I20260808 14:28:45.348034 9279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51458d32cb0e45419694058d98a5260b" candidate_uuid: "b45118b88b48485fa2a5a5814e564c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f017f17924f2495e92d334bf24a622d1" is_pre_election: true
I20260808 14:28:45.348037 9136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51458d32cb0e45419694058d98a5260b" candidate_uuid: "b45118b88b48485fa2a5a5814e564c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7486dac41a34864b4f2327f47fc24ad" is_pre_election: true
I20260808 14:28:45.348351 9136 raft_consensus.cc:2454] T 51458d32cb0e45419694058d98a5260b P c7486dac41a34864b4f2327f47fc24ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b45118b88b48485fa2a5a5814e564c36 in term 0.
I20260808 14:28:45.348351 9279 raft_consensus.cc:2454] T 51458d32cb0e45419694058d98a5260b P f017f17924f2495e92d334bf24a622d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b45118b88b48485fa2a5a5814e564c36 in term 0.
I20260808 14:28:45.348858 8922 leader_election.cc:302] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b45118b88b48485fa2a5a5814e564c36, f017f17924f2495e92d334bf24a622d1; no voters:
I20260808 14:28:45.349165 9359 raft_consensus.cc:2781] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:28:45.349249 9359 raft_consensus.cc:482] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:28:45.349280 9359 raft_consensus.cc:3037] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:45.350366 9359 raft_consensus.cc:504] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } }
I20260808 14:28:45.350750 9359 leader_election.cc:288] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36 [CANDIDATE]: Term 1 election: Requested vote from peers f017f17924f2495e92d334bf24a622d1 (127.7.125.67:40843), c7486dac41a34864b4f2327f47fc24ad (127.7.125.66:45975)
I20260808 14:28:45.351075 9279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51458d32cb0e45419694058d98a5260b" candidate_uuid: "b45118b88b48485fa2a5a5814e564c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f017f17924f2495e92d334bf24a622d1"
I20260808 14:28:45.351109 9136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51458d32cb0e45419694058d98a5260b" candidate_uuid: "b45118b88b48485fa2a5a5814e564c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7486dac41a34864b4f2327f47fc24ad"
I20260808 14:28:45.351222 9279 raft_consensus.cc:3037] T 51458d32cb0e45419694058d98a5260b P f017f17924f2495e92d334bf24a622d1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:45.351236 9136 raft_consensus.cc:3037] T 51458d32cb0e45419694058d98a5260b P c7486dac41a34864b4f2327f47fc24ad [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:45.352329 9279 raft_consensus.cc:2454] T 51458d32cb0e45419694058d98a5260b P f017f17924f2495e92d334bf24a622d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b45118b88b48485fa2a5a5814e564c36 in term 1.
I20260808 14:28:45.352658 8922 leader_election.cc:302] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b45118b88b48485fa2a5a5814e564c36, f017f17924f2495e92d334bf24a622d1; no voters:
I20260808 14:28:45.352763 9136 raft_consensus.cc:2454] T 51458d32cb0e45419694058d98a5260b P c7486dac41a34864b4f2327f47fc24ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b45118b88b48485fa2a5a5814e564c36 in term 1.
I20260808 14:28:45.352869 9359 raft_consensus.cc:2781] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:28:45.353099 9359 raft_consensus.cc:686] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36 [term 1 LEADER]: Becoming Leader. State: Replica: b45118b88b48485fa2a5a5814e564c36, State: Running, Role: LEADER
I20260808 14:28:45.353433 9359 consensus_queue.cc:237] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } }
I20260808 14:28:45.356580 8828 catalog_manager.cc:5495] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36 reported cstate change: term changed from 0 to 1, leader changed from <none> to b45118b88b48485fa2a5a5814e564c36 (127.7.125.65). New cstate: current_term: 1 leader_uuid: "b45118b88b48485fa2a5a5814e564c36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } health_report { overall_health: HEALTHY } } }
W20260808 14:28:45.387271 9327 tablet.cc:2369] T 51458d32cb0e45419694058d98a5260b P f017f17924f2495e92d334bf24a622d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:28:45.395357 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.391993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56048 (local address 127.7.125.65:34545)
0808 14:28:45.392127 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:45.392130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:45.392145 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:28:45.392295 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:45.392297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:45.392352 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:45.392444 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:28:45.392688 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.393183 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.394011 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.394217 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.394295 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.394572 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:28:45.395031 (+ 459us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:28:45.395121 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:45.395200 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 14:28:45.395249 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:28:45.401815 9279 raft_consensus.cc:1261] T 51458d32cb0e45419694058d98a5260b P f017f17924f2495e92d334bf24a622d1 [term 1 FOLLOWER]: Refusing update from remote peer b45118b88b48485fa2a5a5814e564c36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:28:45.402627 9359 consensus_queue.cc:1035] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 14:28:45.404122 9183 tablet.cc:2369] T 51458d32cb0e45419694058d98a5260b P c7486dac41a34864b4f2327f47fc24ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:28:45.404309 9136 raft_consensus.cc:1261] T 51458d32cb0e45419694058d98a5260b P c7486dac41a34864b4f2327f47fc24ad [term 1 FOLLOWER]: Refusing update from remote peer b45118b88b48485fa2a5a5814e564c36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:28:45.404874 9359 consensus_queue.cc:1035] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:28:45.410703 9365 mvcc.cc:205] Tried to move back new op lower bound from 7316272436837347328 to 7316272436650442752. Current Snapshot: MvccSnapshot[applied={T|T < 7316272436837347328}]
I20260808 14:28:45.411209 9366 mvcc.cc:205] Tried to move back new op lower bound from 7316272436837347328 to 7316272436650442752. Current Snapshot: MvccSnapshot[applied={T|T < 7316272436837347328}]
I20260808 14:28:45.423774 8828 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:28:45.424059 8828 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:28:45.426772 8828 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b: Sending DeleteTablet for 3 replicas of tablet 51458d32cb0e45419694058d98a5260b
I20260808 14:28:45.427693 9259 tablet_service.cc:1514] Processing DeleteTablet for tablet 51458d32cb0e45419694058d98a5260b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:28:45 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:59988
I20260808 14:28:45.427841 9116 tablet_service.cc:1514] Processing DeleteTablet for tablet 51458d32cb0e45419694058d98a5260b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:28:45 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:44384
I20260808 14:28:45.427951 8972 tablet_service.cc:1514] Processing DeleteTablet for tablet 51458d32cb0e45419694058d98a5260b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:28:45 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:56038
I20260808 14:28:45.428498 9378 tablet_replica.cc:330] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36: stopping tablet replica
I20260808 14:28:45.428498 9377 tablet_replica.cc:330] T 51458d32cb0e45419694058d98a5260b P c7486dac41a34864b4f2327f47fc24ad: stopping tablet replica
I20260808 14:28:45.428709 8828 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 14:28:45.428818 9377 raft_consensus.cc:2229] T 51458d32cb0e45419694058d98a5260b P c7486dac41a34864b4f2327f47fc24ad [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 14:28:45.429116 8828 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:28:45.429147 9377 raft_consensus.cc:2258] T 51458d32cb0e45419694058d98a5260b P c7486dac41a34864b4f2327f47fc24ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:28:45.429518 9375 tablet_replica.cc:330] T 51458d32cb0e45419694058d98a5260b P f017f17924f2495e92d334bf24a622d1: stopping tablet replica
I20260808 14:28:45.429818 9375 raft_consensus.cc:2229] T 51458d32cb0e45419694058d98a5260b P f017f17924f2495e92d334bf24a622d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:28:45.430106 9375 raft_consensus.cc:2258] T 51458d32cb0e45419694058d98a5260b P f017f17924f2495e92d334bf24a622d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:28:45.431515 9375 ts_tablet_manager.cc:1905] T 51458d32cb0e45419694058d98a5260b P f017f17924f2495e92d334bf24a622d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:28:45.431576 9377 ts_tablet_manager.cc:1905] T 51458d32cb0e45419694058d98a5260b P c7486dac41a34864b4f2327f47fc24ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:28:45.434434 9259 tablet_service.cc:1467] Processing CreateTablet for tablet 5685f440265e48d0a54746c1949fbe6c (DEFAULT_TABLE table=test-table [id=0b343acc37ef4f0aa3da155aa72fcecc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:28:45.434482 9375 ts_tablet_manager.cc:1918] T 51458d32cb0e45419694058d98a5260b P f017f17924f2495e92d334bf24a622d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:28:45.434620 9375 log.cc:1196] T 51458d32cb0e45419694058d98a5260b P f017f17924f2495e92d334bf24a622d1: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/51458d32cb0e45419694058d98a5260b
I20260808 14:28:45.434729 9259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5685f440265e48d0a54746c1949fbe6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:45.434643 9377 ts_tablet_manager.cc:1918] T 51458d32cb0e45419694058d98a5260b P c7486dac41a34864b4f2327f47fc24ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:28:45.434921 9377 log.cc:1196] T 51458d32cb0e45419694058d98a5260b P c7486dac41a34864b4f2327f47fc24ad: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/51458d32cb0e45419694058d98a5260b
I20260808 14:28:45.434968 9375 ts_tablet_manager.cc:1939] T 51458d32cb0e45419694058d98a5260b P f017f17924f2495e92d334bf24a622d1: Deleting consensus metadata
I20260808 14:28:45.435346 9377 ts_tablet_manager.cc:1939] T 51458d32cb0e45419694058d98a5260b P c7486dac41a34864b4f2327f47fc24ad: Deleting consensus metadata
I20260808 14:28:45.436977 9351 tablet_bootstrap.cc:492] T 5685f440265e48d0a54746c1949fbe6c P f017f17924f2495e92d334bf24a622d1: Bootstrap starting.
I20260808 14:28:45.437206 8813 catalog_manager.cc:4841] TS c7486dac41a34864b4f2327f47fc24ad (127.7.125.66:45975): tablet 51458d32cb0e45419694058d98a5260b (table test-table [id=c3961ffcc80f41e48973f53d4c338efd]) successfully deleted
I20260808 14:28:45.437593 8972 tablet_service.cc:1467] Processing CreateTablet for tablet 5685f440265e48d0a54746c1949fbe6c (DEFAULT_TABLE table=test-table [id=0b343acc37ef4f0aa3da155aa72fcecc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:28:45.437883 8972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5685f440265e48d0a54746c1949fbe6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:45.437951 9351 tablet_bootstrap.cc:654] T 5685f440265e48d0a54746c1949fbe6c P f017f17924f2495e92d334bf24a622d1: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:45.439200 9351 tablet_bootstrap.cc:492] T 5685f440265e48d0a54746c1949fbe6c P f017f17924f2495e92d334bf24a622d1: No bootstrap required, opened a new log
I20260808 14:28:45.439285 9351 ts_tablet_manager.cc:1397] T 5685f440265e48d0a54746c1949fbe6c P f017f17924f2495e92d334bf24a622d1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260808 14:28:45.439723 9351 raft_consensus.cc:348] T 5685f440265e48d0a54746c1949fbe6c P f017f17924f2495e92d334bf24a622d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } }
I20260808 14:28:45.439839 9351 raft_consensus.cc:374] T 5685f440265e48d0a54746c1949fbe6c P f017f17924f2495e92d334bf24a622d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:45.439893 9351 raft_consensus.cc:729] T 5685f440265e48d0a54746c1949fbe6c P f017f17924f2495e92d334bf24a622d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f017f17924f2495e92d334bf24a622d1, State: Initialized, Role: FOLLOWER
I20260808 14:28:45.440037 9351 consensus_queue.cc:260] T 5685f440265e48d0a54746c1949fbe6c P f017f17924f2495e92d334bf24a622d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } }
I20260808 14:28:45.440279 9351 ts_tablet_manager.cc:1428] T 5685f440265e48d0a54746c1949fbe6c P f017f17924f2495e92d334bf24a622d1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 14:28:45.440382 9352 tablet_bootstrap.cc:492] T 5685f440265e48d0a54746c1949fbe6c P b45118b88b48485fa2a5a5814e564c36: Bootstrap starting.
I20260808 14:28:45.441239 9352 tablet_bootstrap.cc:654] T 5685f440265e48d0a54746c1949fbe6c P b45118b88b48485fa2a5a5814e564c36: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:45.442584 9352 tablet_bootstrap.cc:492] T 5685f440265e48d0a54746c1949fbe6c P b45118b88b48485fa2a5a5814e564c36: No bootstrap required, opened a new log
I20260808 14:28:45.442669 9352 ts_tablet_manager.cc:1397] T 5685f440265e48d0a54746c1949fbe6c P b45118b88b48485fa2a5a5814e564c36: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 14:28:45.442955 9116 tablet_service.cc:1467] Processing CreateTablet for tablet 5685f440265e48d0a54746c1949fbe6c (DEFAULT_TABLE table=test-table [id=0b343acc37ef4f0aa3da155aa72fcecc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:28:45.443138 9352 raft_consensus.cc:348] T 5685f440265e48d0a54746c1949fbe6c P b45118b88b48485fa2a5a5814e564c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } }
I20260808 14:28:45.443245 9352 raft_consensus.cc:374] T 5685f440265e48d0a54746c1949fbe6c P b45118b88b48485fa2a5a5814e564c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:45.443246 9116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5685f440265e48d0a54746c1949fbe6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:45.443276 9352 raft_consensus.cc:729] T 5685f440265e48d0a54746c1949fbe6c P b45118b88b48485fa2a5a5814e564c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b45118b88b48485fa2a5a5814e564c36, State: Initialized, Role: FOLLOWER
I20260808 14:28:45.443389 9352 consensus_queue.cc:260] T 5685f440265e48d0a54746c1949fbe6c P b45118b88b48485fa2a5a5814e564c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } }
I20260808 14:28:45.443635 9352 ts_tablet_manager.cc:1428] T 5685f440265e48d0a54746c1949fbe6c P b45118b88b48485fa2a5a5814e564c36: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:28:45.445693 9353 tablet_bootstrap.cc:492] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad: Bootstrap starting.
I20260808 14:28:45.445909 8815 catalog_manager.cc:4841] TS f017f17924f2495e92d334bf24a622d1 (127.7.125.67:40843): tablet 51458d32cb0e45419694058d98a5260b (table test-table [id=c3961ffcc80f41e48973f53d4c338efd]) successfully deleted
I20260808 14:28:45.446595 9353 tablet_bootstrap.cc:654] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:45.447089 9378 raft_consensus.cc:2229] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:28:45.447500 9378 raft_consensus.cc:2258] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:28:45.448551 9353 tablet_bootstrap.cc:492] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad: No bootstrap required, opened a new log
I20260808 14:28:45.448665 9353 ts_tablet_manager.cc:1397] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:28:45.448822 9378 ts_tablet_manager.cc:1905] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:28:45.449132 9353 raft_consensus.cc:348] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } }
I20260808 14:28:45.449260 9353 raft_consensus.cc:374] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:45.449295 9353 raft_consensus.cc:729] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7486dac41a34864b4f2327f47fc24ad, State: Initialized, Role: FOLLOWER
I20260808 14:28:45.449412 9353 consensus_queue.cc:260] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } }
I20260808 14:28:45.449697 9353 ts_tablet_manager.cc:1428] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:28:45.451444 9378 ts_tablet_manager.cc:1918] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:28:45.451545 9378 log.cc:1196] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-0/wal/wals/51458d32cb0e45419694058d98a5260b
I20260808 14:28:45.451875 9378 ts_tablet_manager.cc:1939] T 51458d32cb0e45419694058d98a5260b P b45118b88b48485fa2a5a5814e564c36: Deleting consensus metadata
I20260808 14:28:45.452993 8813 catalog_manager.cc:4841] TS b45118b88b48485fa2a5a5814e564c36 (127.7.125.65:34545): tablet 51458d32cb0e45419694058d98a5260b (table test-table [id=c3961ffcc80f41e48973f53d4c338efd]) successfully deleted
I20260808 14:28:45.566740 9358 raft_consensus.cc:482] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:28:45.566941 9358 raft_consensus.cc:504] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } }
I20260808 14:28:45.568243 9358 leader_election.cc:288] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b45118b88b48485fa2a5a5814e564c36 (127.7.125.65:34545), f017f17924f2495e92d334bf24a622d1 (127.7.125.67:40843)
I20260808 14:28:45.572999 9383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.568513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:40843 (local address 127.7.125.66:37705)
0808 14:28:45.568957 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:45.568971 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:45.569060 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:45.569425 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:45.569428 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:45.569450 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:28:45.569571 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.569576 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.570592 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.570596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.571710 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.571720 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.572532 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.572536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.572645 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.572670 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 14:28:45.572745 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 14:28:45.572816 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":56,"thread_start_us":68,"threads_started":1}
I20260808 14:28:45.573000 9347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.568670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:37705 (local address 127.7.125.67:40843)
0808 14:28:45.568828 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:45.568834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:45.569057 (+ 223us) server_negotiation.cc:407] Connection header received
0808 14:28:45.569162 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:45.569166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:45.569219 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:45.569309 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:28:45.569704 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.570499 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.571859 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.572387 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.572519 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.572574 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:45.572779 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 14:28:45.572861 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:28:45.573624 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.568794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:46555 (local address 127.7.125.65:34545)
0808 14:28:45.568924 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:45.568928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:45.568948 (+ 20us) server_negotiation.cc:407] Connection header received
0808 14:28:45.569194 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:45.569200 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:45.569335 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:45.569393 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:28:45.569947 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.570727 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.572302 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.572940 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.573072 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.573132 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:45.573411 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 14:28:45.573499 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 14:28:45.573596 9278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5685f440265e48d0a54746c1949fbe6c" candidate_uuid: "c7486dac41a34864b4f2327f47fc24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f017f17924f2495e92d334bf24a622d1" is_pre_election: true
I20260808 14:28:45.573771 9278 raft_consensus.cc:2454] T 5685f440265e48d0a54746c1949fbe6c P f017f17924f2495e92d334bf24a622d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7486dac41a34864b4f2327f47fc24ad in term 0.
I20260808 14:28:45.574043 9382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.568438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:34545 (local address 127.7.125.66:46555)
0808 14:28:45.568863 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:45.568880 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:45.569010 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:45.569575 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:45.569581 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:45.569607 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:28:45.569736 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.569744 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.570847 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.570851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.572156 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.572167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.573158 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.573162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.573265 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.573287 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 14:28:45.573822 (+ 535us) client_negotiation.cc:240] Negotiation successful
0808 14:28:45.573906 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
I20260808 14:28:45.574151 9069 leader_election.cc:302] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7486dac41a34864b4f2327f47fc24ad, f017f17924f2495e92d334bf24a622d1; no voters:
I20260808 14:28:45.574448 9358 raft_consensus.cc:2781] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:28:45.574515 9358 raft_consensus.cc:482] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:28:45.574546 9358 raft_consensus.cc:3037] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:45.574568 8992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5685f440265e48d0a54746c1949fbe6c" candidate_uuid: "c7486dac41a34864b4f2327f47fc24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b45118b88b48485fa2a5a5814e564c36" is_pre_election: true
I20260808 14:28:45.574822 8992 raft_consensus.cc:2454] T 5685f440265e48d0a54746c1949fbe6c P b45118b88b48485fa2a5a5814e564c36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7486dac41a34864b4f2327f47fc24ad in term 0.
I20260808 14:28:45.575857 9358 raft_consensus.cc:504] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } }
I20260808 14:28:45.576256 9358 leader_election.cc:288] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad [CANDIDATE]: Term 1 election: Requested vote from peers b45118b88b48485fa2a5a5814e564c36 (127.7.125.65:34545), f017f17924f2495e92d334bf24a622d1 (127.7.125.67:40843)
I20260808 14:28:45.576591 8992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5685f440265e48d0a54746c1949fbe6c" candidate_uuid: "c7486dac41a34864b4f2327f47fc24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b45118b88b48485fa2a5a5814e564c36"
I20260808 14:28:45.576627 9278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5685f440265e48d0a54746c1949fbe6c" candidate_uuid: "c7486dac41a34864b4f2327f47fc24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f017f17924f2495e92d334bf24a622d1"
I20260808 14:28:45.576721 8992 raft_consensus.cc:3037] T 5685f440265e48d0a54746c1949fbe6c P b45118b88b48485fa2a5a5814e564c36 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:45.576735 9278 raft_consensus.cc:3037] T 5685f440265e48d0a54746c1949fbe6c P f017f17924f2495e92d334bf24a622d1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:45.577946 8992 raft_consensus.cc:2454] T 5685f440265e48d0a54746c1949fbe6c P b45118b88b48485fa2a5a5814e564c36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7486dac41a34864b4f2327f47fc24ad in term 1.
I20260808 14:28:45.577946 9278 raft_consensus.cc:2454] T 5685f440265e48d0a54746c1949fbe6c P f017f17924f2495e92d334bf24a622d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7486dac41a34864b4f2327f47fc24ad in term 1.
I20260808 14:28:45.578244 9067 leader_election.cc:302] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b45118b88b48485fa2a5a5814e564c36, c7486dac41a34864b4f2327f47fc24ad; no voters:
I20260808 14:28:45.578490 9358 raft_consensus.cc:2781] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:28:45.578770 9358 raft_consensus.cc:686] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad [term 1 LEADER]: Becoming Leader. State: Replica: c7486dac41a34864b4f2327f47fc24ad, State: Running, Role: LEADER
I20260808 14:28:45.579061 9358 consensus_queue.cc:237] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } }
I20260808 14:28:45.581666 8829 catalog_manager.cc:5495] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad reported cstate change: term changed from 0 to 1, leader changed from <none> to c7486dac41a34864b4f2327f47fc24ad (127.7.125.66). New cstate: current_term: 1 leader_uuid: "c7486dac41a34864b4f2327f47fc24ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } health_report { overall_health: UNKNOWN } } }
Aug 08 14:28:45 dist-test-slave-w206 krb5kdc[8784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199324, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:45.629931 9391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.621094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35651 (local address 127.0.0.1:38976)
0808 14:28:45.621423 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:45.621442 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:45.621561 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:45.621806 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:45.621809 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:45.622424 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:45.622573 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.622585 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.623542 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.623544 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.624159 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.624168 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.624626 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.624629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.624691 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.625276 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:45.625288 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:45.626773 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:45.628523 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:45.628531 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:45.628537 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:45.628764 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:45.629015 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:45.629017 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:45.629018 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:45.629096 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:45.629430 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:45.629436 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:45.629538 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 14:28:45.629628 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 14:28:45.629750 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":142,"threads_started":1}
I20260808 14:28:45.629947 8880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.621153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38976 (local address 127.7.125.126:35651)
0808 14:28:45.621275 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:45.621280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:45.621520 (+ 240us) server_negotiation.cc:407] Connection header received
0808 14:28:45.621644 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:45.621647 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:45.621689 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:45.621763 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:45.622738 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.623439 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.624285 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.624517 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.624598 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.626919 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:45.626922 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:45.626925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:45.626946 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:45.628430 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:45.628848 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:45.628850 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:45.628852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:45.628934 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:45.629171 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:45.629173 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:45.629174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:45.629341 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:45.629418 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:45.629686 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 14:28:45.629810 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":42}
I20260808 14:28:45.634184 8829 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:38976:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 14:28:45.635030 8829 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:28:45.640056 8972 tablet_service.cc:1467] Processing CreateTablet for tablet 13d35fc522804125bc5b4f513cb67b7e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=298f3a5b94f5407887f3d073e6b036dd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 14:28:45.640080 9116 tablet_service.cc:1467] Processing CreateTablet for tablet 13d35fc522804125bc5b4f513cb67b7e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=298f3a5b94f5407887f3d073e6b036dd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 14:28:45.640265 9259 tablet_service.cc:1467] Processing CreateTablet for tablet 13d35fc522804125bc5b4f513cb67b7e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=298f3a5b94f5407887f3d073e6b036dd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 14:28:45.640579 8972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13d35fc522804125bc5b4f513cb67b7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:45.640599 9116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13d35fc522804125bc5b4f513cb67b7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:45.640672 9259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13d35fc522804125bc5b4f513cb67b7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:45.643507 9351 tablet_bootstrap.cc:492] T 13d35fc522804125bc5b4f513cb67b7e P f017f17924f2495e92d334bf24a622d1: Bootstrap starting.
I20260808 14:28:45.643579 9352 tablet_bootstrap.cc:492] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36: Bootstrap starting.
I20260808 14:28:45.644250 9353 tablet_bootstrap.cc:492] T 13d35fc522804125bc5b4f513cb67b7e P c7486dac41a34864b4f2327f47fc24ad: Bootstrap starting.
I20260808 14:28:45.644495 9352 tablet_bootstrap.cc:654] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:45.644588 9351 tablet_bootstrap.cc:654] T 13d35fc522804125bc5b4f513cb67b7e P f017f17924f2495e92d334bf24a622d1: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:45.645293 9353 tablet_bootstrap.cc:654] T 13d35fc522804125bc5b4f513cb67b7e P c7486dac41a34864b4f2327f47fc24ad: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:45.645752 9352 tablet_bootstrap.cc:492] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36: No bootstrap required, opened a new log
I20260808 14:28:45.645828 9352 ts_tablet_manager.cc:1397] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 14:28:45.646242 9352 raft_consensus.cc:348] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } }
I20260808 14:28:45.646381 9352 raft_consensus.cc:374] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:45.646422 9352 raft_consensus.cc:729] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b45118b88b48485fa2a5a5814e564c36, State: Initialized, Role: FOLLOWER
I20260808 14:28:45.646554 9352 consensus_queue.cc:260] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } }
I20260808 14:28:45.646660 9353 tablet_bootstrap.cc:492] T 13d35fc522804125bc5b4f513cb67b7e P c7486dac41a34864b4f2327f47fc24ad: No bootstrap required, opened a new log
I20260808 14:28:45.646678 9351 tablet_bootstrap.cc:492] T 13d35fc522804125bc5b4f513cb67b7e P f017f17924f2495e92d334bf24a622d1: No bootstrap required, opened a new log
I20260808 14:28:45.646739 9353 ts_tablet_manager.cc:1397] T 13d35fc522804125bc5b4f513cb67b7e P c7486dac41a34864b4f2327f47fc24ad: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:28:45.646759 9351 ts_tablet_manager.cc:1397] T 13d35fc522804125bc5b4f513cb67b7e P f017f17924f2495e92d334bf24a622d1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:28:45.646893 9352 ts_tablet_manager.cc:1428] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:28:45.647255 9353 raft_consensus.cc:348] T 13d35fc522804125bc5b4f513cb67b7e P c7486dac41a34864b4f2327f47fc24ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } }
I20260808 14:28:45.647277 9351 raft_consensus.cc:348] T 13d35fc522804125bc5b4f513cb67b7e P f017f17924f2495e92d334bf24a622d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } }
I20260808 14:28:45.647367 9353 raft_consensus.cc:374] T 13d35fc522804125bc5b4f513cb67b7e P c7486dac41a34864b4f2327f47fc24ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:45.647320 9371 tablet_replica.cc:439] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } } }
I20260808 14:28:45.647395 9351 raft_consensus.cc:374] T 13d35fc522804125bc5b4f513cb67b7e P f017f17924f2495e92d334bf24a622d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:45.647410 9353 raft_consensus.cc:729] T 13d35fc522804125bc5b4f513cb67b7e P c7486dac41a34864b4f2327f47fc24ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7486dac41a34864b4f2327f47fc24ad, State: Initialized, Role: FOLLOWER
I20260808 14:28:45.647429 9351 raft_consensus.cc:729] T 13d35fc522804125bc5b4f513cb67b7e P f017f17924f2495e92d334bf24a622d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f017f17924f2495e92d334bf24a622d1, State: Initialized, Role: FOLLOWER
I20260808 14:28:45.647450 9371 tablet_replica.cc:442] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:28:45.647531 9353 consensus_queue.cc:260] T 13d35fc522804125bc5b4f513cb67b7e P c7486dac41a34864b4f2327f47fc24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } }
I20260808 14:28:45.647557 9351 consensus_queue.cc:260] T 13d35fc522804125bc5b4f513cb67b7e P f017f17924f2495e92d334bf24a622d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } }
I20260808 14:28:45.647802 9353 ts_tablet_manager.cc:1428] T 13d35fc522804125bc5b4f513cb67b7e P c7486dac41a34864b4f2327f47fc24ad: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:28:45.647881 9351 ts_tablet_manager.cc:1428] T 13d35fc522804125bc5b4f513cb67b7e P f017f17924f2495e92d334bf24a622d1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:28:45.647938 9357 tablet_replica.cc:439] T 13d35fc522804125bc5b4f513cb67b7e P f017f17924f2495e92d334bf24a622d1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } } }
I20260808 14:28:45.647994 9358 tablet_replica.cc:439] T 13d35fc522804125bc5b4f513cb67b7e P c7486dac41a34864b4f2327f47fc24ad: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } } }
I20260808 14:28:45.648082 9357 tablet_replica.cc:442] T 13d35fc522804125bc5b4f513cb67b7e P f017f17924f2495e92d334bf24a622d1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:28:45.648114 9358 tablet_replica.cc:442] T 13d35fc522804125bc5b4f513cb67b7e P c7486dac41a34864b4f2327f47fc24ad: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 14:28:45.677193 9041 tablet.cc:2369] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:28:45.712409 9371 raft_consensus.cc:482] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:28:45.712546 9371 raft_consensus.cc:504] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } }
I20260808 14:28:45.713022 9371 leader_election.cc:288] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7486dac41a34864b4f2327f47fc24ad (127.7.125.66:45975), f017f17924f2495e92d334bf24a622d1 (127.7.125.67:40843)
I20260808 14:28:45.713481 9136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13d35fc522804125bc5b4f513cb67b7e" candidate_uuid: "b45118b88b48485fa2a5a5814e564c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7486dac41a34864b4f2327f47fc24ad" is_pre_election: true
I20260808 14:28:45.713479 9278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13d35fc522804125bc5b4f513cb67b7e" candidate_uuid: "b45118b88b48485fa2a5a5814e564c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f017f17924f2495e92d334bf24a622d1" is_pre_election: true
I20260808 14:28:45.713647 9136 raft_consensus.cc:2454] T 13d35fc522804125bc5b4f513cb67b7e P c7486dac41a34864b4f2327f47fc24ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b45118b88b48485fa2a5a5814e564c36 in term 0.
I20260808 14:28:45.713660 9278 raft_consensus.cc:2454] T 13d35fc522804125bc5b4f513cb67b7e P f017f17924f2495e92d334bf24a622d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b45118b88b48485fa2a5a5814e564c36 in term 0.
I20260808 14:28:45.713974 8920 leader_election.cc:302] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b45118b88b48485fa2a5a5814e564c36, c7486dac41a34864b4f2327f47fc24ad; no voters:
I20260808 14:28:45.714159 9371 raft_consensus.cc:2781] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:28:45.714221 9371 raft_consensus.cc:482] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:28:45.714298 9371 raft_consensus.cc:3037] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:45.715464 9371 raft_consensus.cc:504] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } }
I20260808 14:28:45.715853 9371 leader_election.cc:288] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36 [CANDIDATE]: Term 1 election: Requested vote from peers c7486dac41a34864b4f2327f47fc24ad (127.7.125.66:45975), f017f17924f2495e92d334bf24a622d1 (127.7.125.67:40843)
I20260808 14:28:45.716214 9136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13d35fc522804125bc5b4f513cb67b7e" candidate_uuid: "b45118b88b48485fa2a5a5814e564c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7486dac41a34864b4f2327f47fc24ad"
I20260808 14:28:45.716233 9278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13d35fc522804125bc5b4f513cb67b7e" candidate_uuid: "b45118b88b48485fa2a5a5814e564c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f017f17924f2495e92d334bf24a622d1"
I20260808 14:28:45.716312 9136 raft_consensus.cc:3037] T 13d35fc522804125bc5b4f513cb67b7e P c7486dac41a34864b4f2327f47fc24ad [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:45.716382 9278 raft_consensus.cc:3037] T 13d35fc522804125bc5b4f513cb67b7e P f017f17924f2495e92d334bf24a622d1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:45.717226 9136 raft_consensus.cc:2454] T 13d35fc522804125bc5b4f513cb67b7e P c7486dac41a34864b4f2327f47fc24ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b45118b88b48485fa2a5a5814e564c36 in term 1.
I20260808 14:28:45.717322 9278 raft_consensus.cc:2454] T 13d35fc522804125bc5b4f513cb67b7e P f017f17924f2495e92d334bf24a622d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b45118b88b48485fa2a5a5814e564c36 in term 1.
I20260808 14:28:45.717543 8920 leader_election.cc:302] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b45118b88b48485fa2a5a5814e564c36, c7486dac41a34864b4f2327f47fc24ad; no voters:
I20260808 14:28:45.717705 9371 raft_consensus.cc:2781] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:28:45.717793 9371 raft_consensus.cc:686] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36 [term 1 LEADER]: Becoming Leader. State: Replica: b45118b88b48485fa2a5a5814e564c36, State: Running, Role: LEADER
I20260808 14:28:45.717935 9371 consensus_queue.cc:237] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } }
I20260808 14:28:45.718616 9359 tablet_replica.cc:439] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36: TxnStatusTablet state changed. Reason: New leader b45118b88b48485fa2a5a5814e564c36. Latest consensus state: current_term: 1 leader_uuid: "b45118b88b48485fa2a5a5814e564c36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } } }
I20260808 14:28:45.718725 9359 tablet_replica.cc:442] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36: This TxnStatusTablet replica's current role is: LEADER
I20260808 14:28:45.719053 9396 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 14:28:45.719272 9396 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 14:28:45.719980 8828 catalog_manager.cc:5495] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36 reported cstate change: term changed from 0 to 1, leader changed from <none> to b45118b88b48485fa2a5a5814e564c36 (127.7.125.65). New cstate: current_term: 1 leader_uuid: "b45118b88b48485fa2a5a5814e564c36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } health_report { overall_health: UNKNOWN } } }
I20260808 14:28:45.750651 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.747435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57006 (local address 127.7.125.65:34545)
0808 14:28:45.747591 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:45.747596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:45.747612 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:28:45.747694 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:45.747698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:45.747758 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:45.747844 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:28:45.748122 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.748641 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.749505 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.749736 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.749850 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.750151 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:28:45.750230 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:28:45.750314 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:45.750449 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 14:28:45.750507 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 14:28:45.750651 9391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.747294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:34545 (local address 127.0.0.1:57006)
0808 14:28:45.747455 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:45.747472 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:45.747584 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:45.747871 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:45.747874 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:45.747882 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 14:28:45.748001 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.748006 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.748764 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.748769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.749376 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.749384 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.749853 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.749856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.749960 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.750004 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 14:28:45.750321 (+ 317us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 14:28:45.750340 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 14:28:45.750434 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 14:28:45.750519 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 14:28:45.753612 9136 raft_consensus.cc:1261] T 13d35fc522804125bc5b4f513cb67b7e P c7486dac41a34864b4f2327f47fc24ad [term 1 FOLLOWER]: Refusing update from remote peer b45118b88b48485fa2a5a5814e564c36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:28:45.753603 9278 raft_consensus.cc:1261] T 13d35fc522804125bc5b4f513cb67b7e P f017f17924f2495e92d334bf24a622d1 [term 1 FOLLOWER]: Refusing update from remote peer b45118b88b48485fa2a5a5814e564c36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:28:45.754101 9359 consensus_queue.cc:1035] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:28:45.754287 9371 consensus_queue.cc:1035] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:28:45.755880 9358 tablet_replica.cc:439] T 13d35fc522804125bc5b4f513cb67b7e P c7486dac41a34864b4f2327f47fc24ad: TxnStatusTablet state changed. Reason: New leader b45118b88b48485fa2a5a5814e564c36. Latest consensus state: current_term: 1 leader_uuid: "b45118b88b48485fa2a5a5814e564c36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } } }
I20260808 14:28:45.756026 9358 tablet_replica.cc:442] T 13d35fc522804125bc5b4f513cb67b7e P c7486dac41a34864b4f2327f47fc24ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:28:45.756886 9357 tablet_replica.cc:439] T 13d35fc522804125bc5b4f513cb67b7e P f017f17924f2495e92d334bf24a622d1: TxnStatusTablet state changed. Reason: New leader b45118b88b48485fa2a5a5814e564c36. Latest consensus state: current_term: 1 leader_uuid: "b45118b88b48485fa2a5a5814e564c36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } } }
I20260808 14:28:45.757004 9357 tablet_replica.cc:442] T 13d35fc522804125bc5b4f513cb67b7e P f017f17924f2495e92d334bf24a622d1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:28:45.758050 9359 tablet_replica.cc:439] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b45118b88b48485fa2a5a5814e564c36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } } }
I20260808 14:28:45.758163 9359 tablet_replica.cc:442] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36: This TxnStatusTablet replica's current role is: LEADER
I20260808 14:28:45.760072 9357 tablet_replica.cc:439] T 13d35fc522804125bc5b4f513cb67b7e P f017f17924f2495e92d334bf24a622d1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b45118b88b48485fa2a5a5814e564c36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } } }
I20260808 14:28:45.760176 9357 tablet_replica.cc:442] T 13d35fc522804125bc5b4f513cb67b7e P f017f17924f2495e92d334bf24a622d1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:28:45.763484 9371 tablet_replica.cc:439] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b45118b88b48485fa2a5a5814e564c36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } } }
I20260808 14:28:45.763608 9371 tablet_replica.cc:442] T 13d35fc522804125bc5b4f513cb67b7e P b45118b88b48485fa2a5a5814e564c36: This TxnStatusTablet replica's current role is: LEADER
I20260808 14:28:45.764019 9368 mvcc.cc:205] Tried to move back new op lower bound from 7316272438282608640 to 7316272438142124032. Current Snapshot: MvccSnapshot[applied={T|T < 7316272438282608640}]
I20260808 14:28:45.764268 9358 tablet_replica.cc:439] T 13d35fc522804125bc5b4f513cb67b7e P c7486dac41a34864b4f2327f47fc24ad: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b45118b88b48485fa2a5a5814e564c36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 } } peers { permanent_uuid: "c7486dac41a34864b4f2327f47fc24ad" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 45975 } } peers { permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 } } }
I20260808 14:28:45.764372 9358 tablet_replica.cc:442] T 13d35fc522804125bc5b4f513cb67b7e P c7486dac41a34864b4f2327f47fc24ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:28:45.768740 9349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.764825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35376 (local address 127.7.125.66:45975)
0808 14:28:45.764935 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:45.764949 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:45.764962 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:28:45.765038 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:45.765041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:45.765100 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:45.765204 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:28:45.765465 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.766021 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.767196 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.767423 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.767532 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.767811 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:28:45.768301 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:28:45.768449 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:45.768487 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 14:28:45.768539 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 14:28:45.780207 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.775882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:47265 (local address 127.7.125.65:34545)
0808 14:28:45.776066 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:45.776069 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:45.776159 (+ 90us) server_negotiation.cc:407] Connection header received
0808 14:28:45.776276 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:45.776279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:45.776322 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:45.776386 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:28:45.776665 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.777152 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.779169 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.779382 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.779468 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.779811 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:28:45.779880 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:28:45.779946 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:45.780056 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 14:28:45.780101 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260808 14:28:45.780292 9403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.775720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:34545 (local address 127.7.125.66:47265)
0808 14:28:45.776068 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:45.776083 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:45.776186 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:45.776423 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:45.776426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:45.776434 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 14:28:45.776562 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.776568 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.777254 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.777257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.779059 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.779071 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.779496 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.779500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.779601 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.779653 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 14:28:45.779970 (+ 317us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 14:28:45.779977 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 14:28:45.780048 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 14:28:45.780130 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":142,"threads_started":1}
I20260808 14:28:45.787458 9278 raft_consensus.cc:1261] T 5685f440265e48d0a54746c1949fbe6c P f017f17924f2495e92d334bf24a622d1 [term 1 FOLLOWER]: Refusing update from remote peer c7486dac41a34864b4f2327f47fc24ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:28:45.787662 8992 raft_consensus.cc:1261] T 5685f440265e48d0a54746c1949fbe6c P b45118b88b48485fa2a5a5814e564c36 [term 1 FOLLOWER]: Refusing update from remote peer c7486dac41a34864b4f2327f47fc24ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:28:45.787973 9384 consensus_queue.cc:1035] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "f017f17924f2495e92d334bf24a622d1" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:28:45.788231 9358 consensus_queue.cc:1035] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "b45118b88b48485fa2a5a5814e564c36" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:28:45.811190 9349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.807322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:38579 (local address 127.7.125.66:45975)
0808 14:28:45.807426 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:45.807431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:45.807726 (+ 295us) server_negotiation.cc:407] Connection header received
0808 14:28:45.807906 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:45.807910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:45.807964 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:45.808046 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:28:45.808299 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.808865 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.810049 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.810289 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.810367 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.810716 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:28:45.810797 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:28:45.810857 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:45.810990 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 14:28:45.811071 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 14:28:45.811228 9407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.807291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:45975 (local address 127.7.125.65:38579)
0808 14:28:45.807627 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:45.807663 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:45.807777 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:45.808059 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:45.808063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:45.808071 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 14:28:45.808181 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.808187 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.808991 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.808994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.809931 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:45.809943 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.810400 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.810404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:45.810493 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.810546 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 14:28:45.810887 (+ 341us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 14:28:45.810894 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 14:28:45.810966 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 14:28:45.811073 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":129,"threads_started":1}
I20260808 14:28:45.836018 8829 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:28:45.836194 8829 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:28:45.838539 8829 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 631f45d24031409eafa1d5f91eeda47b: Sending DeleteTablet for 3 replicas of tablet 5685f440265e48d0a54746c1949fbe6c
I20260808 14:28:45.839259 9116 tablet_service.cc:1514] Processing DeleteTablet for tablet 5685f440265e48d0a54746c1949fbe6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:28:45 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:44384
I20260808 14:28:45.839322 9259 tablet_service.cc:1514] Processing DeleteTablet for tablet 5685f440265e48d0a54746c1949fbe6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:28:45 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:59988
I20260808 14:28:45.839536 9377 tablet_replica.cc:330] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad: stopping tablet replica
I20260808 14:28:45.839537 9375 tablet_replica.cc:330] T 5685f440265e48d0a54746c1949fbe6c P f017f17924f2495e92d334bf24a622d1: stopping tablet replica
I20260808 14:28:45.839658 9377 raft_consensus.cc:2229] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:28:45.839680 9375 raft_consensus.cc:2229] T 5685f440265e48d0a54746c1949fbe6c P f017f17924f2495e92d334bf24a622d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:28:45.839787 8972 tablet_service.cc:1514] Processing DeleteTablet for tablet 5685f440265e48d0a54746c1949fbe6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:28:45 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:56038
I20260808 14:28:45.839835 9375 raft_consensus.cc:2258] T 5685f440265e48d0a54746c1949fbe6c P f017f17924f2495e92d334bf24a622d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:28:45.839994 9377 raft_consensus.cc:2258] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:28:45.840029 9378 tablet_replica.cc:330] T 5685f440265e48d0a54746c1949fbe6c P b45118b88b48485fa2a5a5814e564c36: stopping tablet replica
I20260808 14:28:45.840193 9378 raft_consensus.cc:2229] T 5685f440265e48d0a54746c1949fbe6c P b45118b88b48485fa2a5a5814e564c36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:28:45.840322 9378 raft_consensus.cc:2258] T 5685f440265e48d0a54746c1949fbe6c P b45118b88b48485fa2a5a5814e564c36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:28:45.840901 9377 ts_tablet_manager.cc:1905] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:28:45.841099 9375 ts_tablet_manager.cc:1905] T 5685f440265e48d0a54746c1949fbe6c P f017f17924f2495e92d334bf24a622d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:28:45.841182 9378 ts_tablet_manager.cc:1905] T 5685f440265e48d0a54746c1949fbe6c P b45118b88b48485fa2a5a5814e564c36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:28:45.843874 9377 ts_tablet_manager.cc:1918] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:28:45.843874 9378 ts_tablet_manager.cc:1918] T 5685f440265e48d0a54746c1949fbe6c P b45118b88b48485fa2a5a5814e564c36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:28:45.843953 9378 log.cc:1196] T 5685f440265e48d0a54746c1949fbe6c P b45118b88b48485fa2a5a5814e564c36: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-0/wal/wals/5685f440265e48d0a54746c1949fbe6c
I20260808 14:28:45.843951 9377 log.cc:1196] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/5685f440265e48d0a54746c1949fbe6c
I20260808 14:28:45.844252 9377 ts_tablet_manager.cc:1939] T 5685f440265e48d0a54746c1949fbe6c P c7486dac41a34864b4f2327f47fc24ad: Deleting consensus metadata
I20260808 14:28:45.844293 9378 ts_tablet_manager.cc:1939] T 5685f440265e48d0a54746c1949fbe6c P b45118b88b48485fa2a5a5814e564c36: Deleting consensus metadata
I20260808 14:28:45.845016 8813 catalog_manager.cc:4841] TS c7486dac41a34864b4f2327f47fc24ad (127.7.125.66:45975): tablet 5685f440265e48d0a54746c1949fbe6c (table test-table [id=0b343acc37ef4f0aa3da155aa72fcecc]) successfully deleted
I20260808 14:28:45.845240 8813 catalog_manager.cc:4841] TS b45118b88b48485fa2a5a5814e564c36 (127.7.125.65:34545): tablet 5685f440265e48d0a54746c1949fbe6c (table test-table [id=0b343acc37ef4f0aa3da155aa72fcecc]) successfully deleted
I20260808 14:28:45.845538 9375 ts_tablet_manager.cc:1918] T 5685f440265e48d0a54746c1949fbe6c P f017f17924f2495e92d334bf24a622d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:28:45.845645 9375 log.cc:1196] T 5685f440265e48d0a54746c1949fbe6c P f017f17924f2495e92d334bf24a622d1: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/5685f440265e48d0a54746c1949fbe6c
I20260808 14:28:45.845968 9375 ts_tablet_manager.cc:1939] T 5685f440265e48d0a54746c1949fbe6c P f017f17924f2495e92d334bf24a622d1: Deleting consensus metadata
I20260808 14:28:45.846897 8815 catalog_manager.cc:4841] TS f017f17924f2495e92d334bf24a622d1 (127.7.125.67:40843): tablet 5685f440265e48d0a54746c1949fbe6c (table test-table [id=0b343acc37ef4f0aa3da155aa72fcecc]) successfully deleted
Aug 08 14:28:45 dist-test-slave-w206 krb5kdc[8784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199325, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.125.65@KRBTEST.COM
I20260808 14:28:45.860479 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.847129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57020 (local address 127.7.125.65:34545)
0808 14:28:45.847285 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:45.847290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:45.847664 (+ 374us) server_negotiation.cc:407] Connection header received
0808 14:28:45.847913 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:45.847919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:45.847971 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:45.848021 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:45.849015 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.849835 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.851097 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.851425 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.851502 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.855951 (+ 4449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:45.855967 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:45.855986 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:45.856017 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:45.858205 (+ 2188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:45.858738 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:45.858746 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:45.858748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:45.858848 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:45.859184 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:45.859187 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:45.859189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:45.859506 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:45.859722 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:45.860169 (+ 447us) server_negotiation.cc:299] Negotiation successful
0808 14:28:45.860305 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260808 14:28:45.861873 9033 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57020
I20260808 14:28:45.873912 8880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:45.863970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35976 (local address 127.7.125.126:35651)
0808 14:28:45.864093 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:45.864098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:45.864172 (+ 74us) server_negotiation.cc:407] Connection header received
0808 14:28:45.864353 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:45.864356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:45.864410 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:45.864462 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:45.865348 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.866182 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.867435 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:45.867784 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:45.867858 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:45.869978 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:45.869982 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:45.869985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:45.870013 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:45.871962 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:45.872515 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:45.872518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:45.872520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:45.872614 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:45.872938 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:45.872941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:45.872943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:45.873163 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:45.873296 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:45.873600 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 14:28:45.873738 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 14:28:45.875293 8829 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35976
I20260808 14:28:45.876797 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 8903
I20260808 14:28:45.886734 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 9050
I20260808 14:28:45.894506 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 9191
I20260808 14:28:45.902738 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 8797
2026-08-08T14:28:45Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2943 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:28:45 dist-test-slave-w206 krb5kdc[9418](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:28:45 dist-test-slave-w206 krb5kdc[9418](info): set up 2 sockets
Aug 08 14:28:45 dist-test-slave-w206 krb5kdc[9418](info): commencing operation
krb5kdc: starting...
W20260808 14:28:47.042723 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:28:47 dist-test-slave-w206 krb5kdc[9418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:28:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:28:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:28:47.174691 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:43091
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:43091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:47.286072 9432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:47.286329 9432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:47.286367 9432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:47.290293 9432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:28:47.290347 9432 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 14:28:47.290364 9432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:47.290382 9432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:47.290400 9432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:28:47.290414 9432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:28:47.295660 9432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:43091
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:43091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:47.296631 9432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:47.297602 9432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:28:47.306329 9432 server_base.cc:1034] running on GCE node
W20260808 14:28:47.306329 9439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:47.306330 9437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:47.306557 9441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:47.307179 9432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:47.308228 9432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:47.309389 9432 hybrid_clock.cc:648] HybridClock initialized: now 1786199327309371 us; error 29 us; skew 500 ppm
Aug 08 14:28:47 dist-test-slave-w206 krb5kdc[9418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199327, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:47.317437 9432 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:28:47.318411 9432 webserver.cc:460] Webserver started at http://127.7.125.126:33427/ using document root <none> and password file <none>
I20260808 14:28:47.318941 9432 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:47.319058 9432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:47.319253 9432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:47.321343 9432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "b58df962d3fe423ab32a8136a5e1b79f"
format_stamp: "Formatted at 2026-08-08 14:28:47 on dist-test-slave-w206"
server_key: "d373911ded155a9c2a8d17416c364df7"
server_key_iv: "398b9571cb7cf11c4765c7e66d85c468"
server_key_version: "encryptionkey@0"
I20260808 14:28:47.321866 9432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "b58df962d3fe423ab32a8136a5e1b79f"
format_stamp: "Formatted at 2026-08-08 14:28:47 on dist-test-slave-w206"
server_key: "d373911ded155a9c2a8d17416c364df7"
server_key_iv: "398b9571cb7cf11c4765c7e66d85c468"
server_key_version: "encryptionkey@0"
I20260808 14:28:47.325591 9432 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:28:47.328027 9449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:47.329250 9432 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:28:47.329411 9432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "b58df962d3fe423ab32a8136a5e1b79f"
format_stamp: "Formatted at 2026-08-08 14:28:47 on dist-test-slave-w206"
server_key: "d373911ded155a9c2a8d17416c364df7"
server_key_iv: "398b9571cb7cf11c4765c7e66d85c468"
server_key_version: "encryptionkey@0"
I20260808 14:28:47.329520 9432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:47.339249 9432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:47.342054 9432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:47.342260 9432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:47.350361 9432 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:43091
I20260808 14:28:47.350394 9512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:43091 every 8 connection(s)
I20260808 14:28:47.351361 9432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:28:47.351629 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 9432
I20260808 14:28:47.351755 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:28:47.352079 7669 external_mini_cluster.cc:1428] Setting key f959bb37c73f70b600a73d6b461c67dd
I20260808 14:28:47.354498 9513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:47.360070 9513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f: Bootstrap starting.
Aug 08 14:28:47 dist-test-slave-w206 krb5kdc[9418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:47.362517 9513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:47.363785 9513 log.cc:824] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f: Log is configured to *not* fsync() on all Append() calls
I20260808 14:28:47.365794 9513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f: No bootstrap required, opened a new log
I20260808 14:28:47.366127 9517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:47.353515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37000 (local address 127.7.125.126:43091)
0808 14:28:47.353955 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:47.353970 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:47.354006 (+ 36us) server_negotiation.cc:407] Connection header received
0808 14:28:47.354689 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:47.354706 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:47.355086 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:47.355421 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:47.356216 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:47.357197 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:47.358055 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:47.358532 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:47.358706 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:47.361412 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:47.361417 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:47.361430 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:47.361456 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:47.363272 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:47.363734 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:47.363739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:47.363744 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:47.363860 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:47.364178 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:47.364182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:47.364184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:47.364589 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:47.364792 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:47.365089 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 14:28:47.365381 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":141,"threads_started":1}
I20260808 14:28:47.369292 9513 raft_consensus.cc:348] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58df962d3fe423ab32a8136a5e1b79f" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43091 } }
I20260808 14:28:47.369529 9513 raft_consensus.cc:374] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:47.369611 9513 raft_consensus.cc:729] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b58df962d3fe423ab32a8136a5e1b79f, State: Initialized, Role: FOLLOWER
I20260808 14:28:47.370082 9513 consensus_queue.cc:260] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58df962d3fe423ab32a8136a5e1b79f" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43091 } }
I20260808 14:28:47.370267 9513 raft_consensus.cc:388] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:28:47.370342 9513 raft_consensus.cc:482] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:28:47.370441 9513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:47.371397 9513 raft_consensus.cc:504] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58df962d3fe423ab32a8136a5e1b79f" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43091 } }
I20260808 14:28:47.371733 9513 leader_election.cc:302] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b58df962d3fe423ab32a8136a5e1b79f; no voters:
I20260808 14:28:47.372037 9513 leader_election.cc:288] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:28:47.372308 9519 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:28:47.372547 9519 raft_consensus.cc:686] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f [term 1 LEADER]: Becoming Leader. State: Replica: b58df962d3fe423ab32a8136a5e1b79f, State: Running, Role: LEADER
I20260808 14:28:47.372864 9519 consensus_queue.cc:237] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58df962d3fe423ab32a8136a5e1b79f" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43091 } }
I20260808 14:28:47.373263 9513 sys_catalog.cc:564] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:28:47.374493 9522 sys_catalog.cc:455] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b58df962d3fe423ab32a8136a5e1b79f. Latest consensus state: current_term: 1 leader_uuid: "b58df962d3fe423ab32a8136a5e1b79f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58df962d3fe423ab32a8136a5e1b79f" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43091 } } }
I20260808 14:28:47.374636 9522 sys_catalog.cc:458] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f [sys.catalog]: This master's current role is: LEADER
I20260808 14:28:47.375015 9521 sys_catalog.cc:455] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b58df962d3fe423ab32a8136a5e1b79f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58df962d3fe423ab32a8136a5e1b79f" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43091 } } }
I20260808 14:28:47.375138 9521 sys_catalog.cc:458] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f [sys.catalog]: This master's current role is: LEADER
W20260808 14:28:47.378273 9535 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:28:47.378341 9535 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:28:47.378423 9529 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:28:47.379206 9529 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:28:47.384706 9529 catalog_manager.cc:1347] Generated new cluster ID: 0a6e2807aaeb41bda7628143c1bc84d6
I20260808 14:28:47.384775 9529 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:28:47.390841 9529 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:28:47.392040 9529 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:28:47.398563 9529 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f: Generated new TSK 0
I20260808 14:28:47.399415 9529 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.125.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
I20260808 14:28:47.437754 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:43091
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:47.551860 9541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:47.552120 9541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:47.552201 9541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:47.556262 9541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:47.556324 9541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:47.556398 9541 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 14:28:47.556438 9541 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 14:28:47.556499 9541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:28:47.561868 9541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.125.126:43091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:47.563002 9541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:47.564064 9541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:28:47.573967 9546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:47.573985 9549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:47.573961 9547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:47.574160 9541 server_base.cc:1034] running on GCE node
I20260808 14:28:47.574617 9541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:47.575405 9541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:47.576594 9541 hybrid_clock.cc:648] HybridClock initialized: now 1786199327576577 us; error 33 us; skew 500 ppm
Aug 08 14:28:47 dist-test-slave-w206 krb5kdc[9418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199327, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:47.579550 9541 init.cc:377] Logged in from keytab as kudu/127.7.125.65@KRBTEST.COM (short username kudu)
I20260808 14:28:47.580437 9541 webserver.cc:460] Webserver started at http://127.7.125.65:42371/ using document root <none> and password file <none>
I20260808 14:28:47.581007 9541 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:47.581099 9541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:47.581347 9541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:47.583467 9541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "e4882374507a4151ba119338ce4daafc"
format_stamp: "Formatted at 2026-08-08 14:28:47 on dist-test-slave-w206"
server_key: "b073a5a6f5afaf2e80ad75aeb67cc17c"
server_key_iv: "e349122fb0cb09925e054895428483c4"
server_key_version: "encryptionkey@0"
I20260808 14:28:47.584167 9541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "e4882374507a4151ba119338ce4daafc"
format_stamp: "Formatted at 2026-08-08 14:28:47 on dist-test-slave-w206"
server_key: "b073a5a6f5afaf2e80ad75aeb67cc17c"
server_key_iv: "e349122fb0cb09925e054895428483c4"
server_key_version: "encryptionkey@0"
I20260808 14:28:47.587895 9541 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:28:47.590215 9556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:47.591449 9541 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:28:47.591583 9541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "e4882374507a4151ba119338ce4daafc"
format_stamp: "Formatted at 2026-08-08 14:28:47 on dist-test-slave-w206"
server_key: "b073a5a6f5afaf2e80ad75aeb67cc17c"
server_key_iv: "e349122fb0cb09925e054895428483c4"
server_key_version: "encryptionkey@0"
I20260808 14:28:47.591697 9541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:47.605014 9541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:47.608455 9541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:47.608664 9541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:47.610082 9541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:28:47.610157 9541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:47.610256 9541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:28:47.610307 9541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 14:28:47 dist-test-slave-w206 krb5kdc[9418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199327, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:47.622016 9541 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:35287
I20260808 14:28:47.622175 9678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:35287 every 8 connection(s)
I20260808 14:28:47.623180 9541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:28:47.624765 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 9541
I20260808 14:28:47.624893 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:28:47.625167 7669 external_mini_cluster.cc:1428] Setting key 9a598f8cdf858504aa875f849c56eb56
I20260808 14:28:47.625329 9517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:47.611059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:57263 (local address 127.7.125.126:43091)
0808 14:28:47.611189 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:47.611192 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:47.612205 (+ 1013us) server_negotiation.cc:407] Connection header received
0808 14:28:47.613194 (+ 989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:47.613198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:47.613257 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:47.613317 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:47.615067 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:47.615646 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:47.617188 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:47.617763 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:47.617879 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:47.621085 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:47.621088 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:47.621090 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:47.621115 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:47.622880 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:47.623555 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:47.623560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:47.623562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:47.623666 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:47.624113 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:47.624117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:47.624118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:47.624399 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:47.624494 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:47.625004 (+ 510us) server_negotiation.cc:299] Negotiation successful
0808 14:28:47.625140 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 14:28:47.626201 9565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:47.611578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43091 (local address 127.7.125.65:57263)
0808 14:28:47.612056 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:47.612097 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:47.612932 (+ 835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:47.613766 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:47.613790 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:47.614303 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:47.614908 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:47.614928 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:47.615780 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:47.615783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:47.617064 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:47.617079 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:47.617922 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:47.617926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:47.618134 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:47.618816 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:47.618836 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:47.620902 (+ 2066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:47.623058 (+ 2156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:47.623064 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:47.623083 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:47.623424 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:47.623796 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:47.623802 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:47.623804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:47.623988 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:47.624505 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:47.624530 (+ 25us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:47.624749 (+ 219us) client_negotiation.cc:769] Sending connection context
0808 14:28:47.624940 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 14:28:47.625251 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":128,"threads_started":1}
I20260808 14:28:47.627602 9679 heartbeater.cc:347] Connected to a master server at 127.7.125.126:43091
I20260808 14:28:47.627734 9679 heartbeater.cc:464] Registering TS with master...
I20260808 14:28:47.628281 9679 heartbeater.cc:511] Master 127.7.125.126:43091 requested a full tablet report, sending...
I20260808 14:28:47.629920 9466 ts_manager.cc:194] Registered new tserver with Master: e4882374507a4151ba119338ce4daafc (127.7.125.65:35287)
I20260808 14:28:47.631486 9466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:57263
WARNING: no policy specified for kudu/127.7.125.66@KRBTEST.COM; defaulting to no policy
I20260808 14:28:47.638186 9517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:47.630130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:58347 (local address 127.7.125.126:43091)
0808 14:28:47.630268 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:47.630275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:47.630476 (+ 201us) server_negotiation.cc:407] Connection header received
0808 14:28:47.630571 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:47.630573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:47.630609 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:47.630674 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:47.631211 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:47.631915 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:47.633043 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:47.633329 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:47.633439 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:47.634690 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:47.634694 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:47.634696 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:47.634719 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:47.636303 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:47.636794 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:47.636800 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:47.636801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:47.636895 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:47.637264 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:47.637267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:47.637269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:47.637461 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:47.637578 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:47.637878 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 14:28:47.638005 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 14:28:47.638248 9686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:47.630041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43091 (local address 127.7.125.65:58347)
0808 14:28:47.630382 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:47.630397 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:47.630492 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:47.630722 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:47.630725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:47.630942 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:47.631081 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:47.631088 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:47.632078 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:47.632081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:47.632926 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:47.632935 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:47.633443 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:47.633446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:47.633542 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:47.634139 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:47.634160 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:47.634562 (+ 402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:47.636445 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:47.636450 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:47.636453 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:47.636682 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:47.637007 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:47.637010 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:47.637012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:47.637141 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:47.637579 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:47.637583 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:47.637691 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 14:28:47.637806 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 14:28:47.638007 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":87,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
I20260808 14:28:47.658502 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:43091
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:47.769961 9688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:47.770241 9688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:47.770328 9688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:47.774510 9688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:47.774592 9688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:47.774659 9688 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 14:28:47.774714 9688 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 14:28:47.774784 9688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:28:47.780031 9688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.125.126:43091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:47.781111 9688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:47.782150 9688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:28:47.791792 9693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:47.791819 9696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:47.791797 9694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:47.792125 9688 server_base.cc:1034] running on GCE node
I20260808 14:28:47.792596 9688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:47.793210 9688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:47.794373 9688 hybrid_clock.cc:648] HybridClock initialized: now 1786199327794359 us; error 23 us; skew 500 ppm
Aug 08 14:28:47 dist-test-slave-w206 krb5kdc[9418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199327, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:47.797240 9688 init.cc:377] Logged in from keytab as kudu/127.7.125.66@KRBTEST.COM (short username kudu)
I20260808 14:28:47.798146 9688 webserver.cc:460] Webserver started at http://127.7.125.66:36925/ using document root <none> and password file <none>
I20260808 14:28:47.798705 9688 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:47.798799 9688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:47.799093 9688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:47.801122 9688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "a50fe1e1869147b7adacc0b0a821006a"
format_stamp: "Formatted at 2026-08-08 14:28:47 on dist-test-slave-w206"
server_key: "b680510ea2c9790db88d478c4d702cb9"
server_key_iv: "526cce8b0902767bccad809abdb8a626"
server_key_version: "encryptionkey@0"
I20260808 14:28:47.801693 9688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "a50fe1e1869147b7adacc0b0a821006a"
format_stamp: "Formatted at 2026-08-08 14:28:47 on dist-test-slave-w206"
server_key: "b680510ea2c9790db88d478c4d702cb9"
server_key_iv: "526cce8b0902767bccad809abdb8a626"
server_key_version: "encryptionkey@0"
I20260808 14:28:47.805586 9688 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:28:47.808238 9703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:47.809455 9688 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:28:47.809594 9688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "a50fe1e1869147b7adacc0b0a821006a"
format_stamp: "Formatted at 2026-08-08 14:28:47 on dist-test-slave-w206"
server_key: "b680510ea2c9790db88d478c4d702cb9"
server_key_iv: "526cce8b0902767bccad809abdb8a626"
server_key_version: "encryptionkey@0"
I20260808 14:28:47.809711 9688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:47.830137 9688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:47.833134 9688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:47.833348 9688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:47.834916 9688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:28:47.835017 9688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:47.835089 9688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:28:47.835144 9688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 14:28:47 dist-test-slave-w206 krb5kdc[9418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199327, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:47.846458 9688 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:43119
I20260808 14:28:47.846469 9824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:43119 every 8 connection(s)
I20260808 14:28:47.847661 9688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:28:47.849251 9517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:47.835894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:52753 (local address 127.7.125.126:43091)
0808 14:28:47.836077 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:47.836083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:47.836892 (+ 809us) server_negotiation.cc:407] Connection header received
0808 14:28:47.837924 (+ 1032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:47.837927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:47.837986 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:47.838069 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:47.839399 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:47.839935 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:47.841475 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:47.841738 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:47.841853 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:47.845079 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:47.845083 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:47.845086 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:47.845114 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:47.846843 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:47.847442 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:47.847445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:47.847447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:47.847547 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:47.848018 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:47.848025 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:47.848030 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:47.848287 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:47.848407 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:47.849018 (+ 611us) server_negotiation.cc:299] Negotiation successful
0808 14:28:47.849133 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 14:28:47.850067 9712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:47.836260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43091 (local address 127.7.125.66:52753)
0808 14:28:47.836717 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:47.836768 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:47.837700 (+ 932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:47.838304 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:47.838314 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:47.838684 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:47.839234 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:47.839248 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:47.840067 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:47.840070 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:47.841348 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:47.841358 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:47.841908 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:47.841912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:47.842086 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:47.842836 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:47.842864 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:47.844913 (+ 2049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:47.847015 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:47.847023 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:47.847040 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:47.847324 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:47.847650 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:47.847652 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:47.847653 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:47.847895 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:47.848468 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:47.848475 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:47.848779 (+ 304us) client_negotiation.cc:769] Sending connection context
0808 14:28:47.848935 (+ 156us) client_negotiation.cc:240] Negotiation successful
0808 14:28:47.849185 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":125,"threads_started":1}
I20260808 14:28:47.851323 9825 heartbeater.cc:347] Connected to a master server at 127.7.125.126:43091
I20260808 14:28:47.851480 9825 heartbeater.cc:464] Registering TS with master...
I20260808 14:28:47.851964 9825 heartbeater.cc:511] Master 127.7.125.126:43091 requested a full tablet report, sending...
I20260808 14:28:47.853032 9465 ts_manager.cc:194] Registered new tserver with Master: a50fe1e1869147b7adacc0b0a821006a (127.7.125.66:43119)
I20260808 14:28:47.853888 9465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:52753
I20260808 14:28:47.855275 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 9688
I20260808 14:28:47.855401 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:28:47.855705 7669 external_mini_cluster.cc:1428] Setting key 9caa7b2488e3532792a76da6675a0693
I20260808 14:28:47.861025 9517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:47.853555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:51393 (local address 127.7.125.126:43091)
0808 14:28:47.853685 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:47.853691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:47.853828 (+ 137us) server_negotiation.cc:407] Connection header received
0808 14:28:47.853941 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:47.853943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:47.854004 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:47.854122 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:47.854575 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:47.855128 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:47.856132 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:47.856376 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:47.856480 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:47.857875 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:47.857877 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:47.857879 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:47.857912 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:47.859345 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:47.859783 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:47.859789 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:47.859791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:47.859871 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:47.860190 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:47.860194 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:47.860195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:47.860402 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:47.860515 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:47.860773 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 14:28:47.860881 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 14:28:47.861040 9831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:47.853411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43091 (local address 127.7.125.66:51393)
0808 14:28:47.853742 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:47.853756 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:47.853851 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:47.854138 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:47.854141 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:47.854338 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:47.854454 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:47.854462 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:47.855250 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:47.855253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:47.856009 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:47.856019 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:47.856491 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:47.856494 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:47.856579 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:47.857262 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:47.857278 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:47.857739 (+ 461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:47.859462 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:47.859467 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:47.859469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:47.859676 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:47.860001 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:47.860004 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:47.860006 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:47.860097 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:47.860505 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:47.860508 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:47.860609 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 14:28:47.860718 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 14:28:47.860859 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":95,"threads_started":1}
WARNING: no policy specified for kudu/127.7.125.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
I20260808 14:28:47.885938 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:43091
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:47.998513 9834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:47.998765 9834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:47.998811 9834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:48.002813 9834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:48.002879 9834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:48.002920 9834 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 14:28:48.002934 9834 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 14:28:48.002966 9834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:28:48.008587 9834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.125.126:43091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:48.009586 9834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:48.010581 9834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:28:48.020504 9842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:48.020483 9840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:48.020483 9839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:48.020780 9834 server_base.cc:1034] running on GCE node
I20260808 14:28:48.021272 9834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:48.021922 9834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:48.023104 9834 hybrid_clock.cc:648] HybridClock initialized: now 1786199328023091 us; error 24 us; skew 500 ppm
Aug 08 14:28:48 dist-test-slave-w206 krb5kdc[9418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199328, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:48.025835 9834 init.cc:377] Logged in from keytab as kudu/127.7.125.67@KRBTEST.COM (short username kudu)
I20260808 14:28:48.026757 9834 webserver.cc:460] Webserver started at http://127.7.125.67:43109/ using document root <none> and password file <none>
I20260808 14:28:48.027390 9834 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:48.027484 9834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:48.027714 9834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:48.029743 9834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "ad9701502ca34fa0b897099cca03d11d"
format_stamp: "Formatted at 2026-08-08 14:28:48 on dist-test-slave-w206"
server_key: "9dc1e110384000c46b0bdde3a5a08510"
server_key_iv: "2fce563d503d0c7ef0ef475b94649c8d"
server_key_version: "encryptionkey@0"
I20260808 14:28:48.030304 9834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "ad9701502ca34fa0b897099cca03d11d"
format_stamp: "Formatted at 2026-08-08 14:28:48 on dist-test-slave-w206"
server_key: "9dc1e110384000c46b0bdde3a5a08510"
server_key_iv: "2fce563d503d0c7ef0ef475b94649c8d"
server_key_version: "encryptionkey@0"
I20260808 14:28:48.034262 9834 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:28:48.036666 9849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:48.037864 9834 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:28:48.037997 9834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "ad9701502ca34fa0b897099cca03d11d"
format_stamp: "Formatted at 2026-08-08 14:28:48 on dist-test-slave-w206"
server_key: "9dc1e110384000c46b0bdde3a5a08510"
server_key_iv: "2fce563d503d0c7ef0ef475b94649c8d"
server_key_version: "encryptionkey@0"
I20260808 14:28:48.038107 9834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:48.067659 9834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:48.070587 9834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:48.070811 9834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:48.072487 9834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:28:48.072563 9834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:48.072633 9834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:28:48.072670 9834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 14:28:48 dist-test-slave-w206 krb5kdc[9418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199328, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:48.083925 9834 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:39819
I20260808 14:28:48.084327 9970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:39819 every 8 connection(s)
I20260808 14:28:48.084882 9834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:28:48.087605 9517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.073248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:50563 (local address 127.7.125.126:43091)
0808 14:28:48.073378 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:48.073381 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:48.074565 (+ 1184us) server_negotiation.cc:407] Connection header received
0808 14:28:48.075731 (+ 1166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:48.075735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:48.075796 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:48.075889 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:48.077597 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.078469 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.080439 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.080677 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.080767 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.083675 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:48.083679 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:48.083682 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:48.083718 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:48.085553 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:48.086076 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:48.086079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:48.086080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:48.086162 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:48.086474 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:48.086476 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:48.086477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:48.086655 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:48.086729 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:48.087295 (+ 566us) server_negotiation.cc:299] Negotiation successful
0808 14:28:48.087433 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 14:28:48.088349 9858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.073799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43091 (local address 127.7.125.67:50563)
0808 14:28:48.074414 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:48.074460 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:48.075511 (+ 1051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:48.076126 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:48.076137 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:48.076687 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:48.077404 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.077427 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.078619 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.078624 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.080298 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.080311 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.080848 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.080854 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.081078 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.081827 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:48.081847 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:48.083491 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:48.085693 (+ 2202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:48.085698 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:48.085712 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:48.085979 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:48.086248 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:48.086250 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:48.086251 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:48.086382 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:48.086741 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:48.086745 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:48.087022 (+ 277us) client_negotiation.cc:769] Sending connection context
0808 14:28:48.087231 (+ 209us) client_negotiation.cc:240] Negotiation successful
0808 14:28:48.087553 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":127,"threads_started":1}
I20260808 14:28:48.089515 9971 heartbeater.cc:347] Connected to a master server at 127.7.125.126:43091
I20260808 14:28:48.089675 9971 heartbeater.cc:464] Registering TS with master...
I20260808 14:28:48.090163 9971 heartbeater.cc:511] Master 127.7.125.126:43091 requested a full tablet report, sending...
I20260808 14:28:48.091282 9465 ts_manager.cc:194] Registered new tserver with Master: ad9701502ca34fa0b897099cca03d11d (127.7.125.67:39819)
I20260808 14:28:48.091993 9465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:50563
I20260808 14:28:48.093405 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 9834
I20260808 14:28:48.093521 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:28:48.093851 7669 external_mini_cluster.cc:1428] Setting key b7ebcb3a126a2aee4121f7c98f8aaf3a
I20260808 14:28:48.096004 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:28:48.099198 9977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.091744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43091 (local address 127.7.125.67:55115)
0808 14:28:48.092057 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:48.092074 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:48.092207 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:48.092449 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:48.092452 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:48.092653 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:48.092752 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.092758 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.093498 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.093502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.094103 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.094130 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.094684 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.094687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.094777 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.095343 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:48.095354 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:48.095767 (+ 413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:48.097525 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:48.097531 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:48.097536 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:48.097811 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:48.098077 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:48.098083 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:48.098088 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:48.098197 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:48.098674 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:48.098678 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:48.098781 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 14:28:48.098881 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 14:28:48.099047 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":1,"thread_start_us":92,"threads_started":1}
I20260808 14:28:48.099324 9517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.091880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:55115 (local address 127.7.125.126:43091)
0808 14:28:48.091996 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:48.092000 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:48.092140 (+ 140us) server_negotiation.cc:407] Connection header received
0808 14:28:48.092292 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:48.092295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:48.092340 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:48.092433 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:48.092868 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.093362 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.094246 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.094568 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.094671 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.095894 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:48.095899 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:48.095901 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:48.095928 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:48.097413 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:48.097910 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:48.097913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:48.097914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:48.097987 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:48.098314 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:48.098318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:48.098321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:48.098561 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:48.098675 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:48.099007 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 14:28:48.099145 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 14:28:48 dist-test-slave-w206 krb5kdc[9418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 14:28:48 dist-test-slave-w206 krb5kdc[9418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:48.124589 9517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.116125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37010 (local address 127.7.125.126:43091)
0808 14:28:48.116263 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:48.116267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:48.116280 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:28:48.116323 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:48.116325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:48.116366 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:48.116441 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:48.117225 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.117735 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.118571 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.118777 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.118850 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.121321 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:48.121325 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:48.121327 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:48.121352 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:48.122811 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:48.123372 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:48.123374 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:48.123376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:48.123456 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:48.123732 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:48.123734 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:48.123735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:48.123924 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:48.124028 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:48.124286 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 14:28:48.124384 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 14:28:48.127871 9465 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37010:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:28:48.131174 9465 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:28:48.146471 9992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.140794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40724 (local address 127.7.125.65:35287)
0808 14:28:48.141125 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:48.141132 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:48.141623 (+ 491us) server_negotiation.cc:407] Connection header received
0808 14:28:48.141780 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:48.141785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:48.141934 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:48.142072 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:28:48.142515 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.143405 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.145163 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.145723 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.145872 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.146027 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:48.146217 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 14:28:48.146316 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":80,"threads_started":1}
I20260808 14:28:48.146524 9991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.140460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:35287 (local address 127.0.0.1:40724)
0808 14:28:48.141179 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:48.141194 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:48.141300 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:48.142215 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:48.142218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:48.142235 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:28:48.142371 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.142381 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.143516 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.143522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.145006 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.145018 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.145882 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.145886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.146002 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.146041 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 14:28:48.146174 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 14:28:48.146302 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"mutex_wait_us":42,"thread_start_us":94,"threads_started":1}
I20260808 14:28:48.146848 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.141112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55388 (local address 127.7.125.67:39819)
0808 14:28:48.141869 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:48.141882 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:48.141905 (+ 23us) server_negotiation.cc:407] Connection header received
0808 14:28:48.141978 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:48.141983 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:48.142163 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:48.142322 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:28:48.142537 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.143673 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.145166 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.145943 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.146061 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.146214 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:48.146621 (+ 407us) server_negotiation.cc:299] Negotiation successful
0808 14:28:48.146709 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":127,"threads_started":1}
I20260808 14:28:48.146559 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.140090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:43119 (local address 127.0.0.1:46398)
0808 14:28:48.140441 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:48.140495 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:48.140628 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:48.142065 (+ 1437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:48.142072 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:48.142095 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:28:48.142239 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.142251 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.143783 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.143787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.144968 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.144978 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.146087 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.146091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.146176 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.146207 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 14:28:48.146259 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 14:28:48.146354 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":48,"thread_start_us":105,"threads_started":1}
I20260808 14:28:48.147132 9990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.140285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46398 (local address 127.7.125.66:43119)
0808 14:28:48.141423 (+ 1138us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:48.141437 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:48.141508 (+ 71us) server_negotiation.cc:407] Connection header received
0808 14:28:48.141589 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:48.141597 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:48.141871 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:48.142073 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:28:48.142379 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.143663 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.145097 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.145943 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.146634 (+ 691us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.146782 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:48.146827 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 14:28:48.146921 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1035,"thread_start_us":122,"threads_started":1}
I20260808 14:28:48.147073 9993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.140922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:39819 (local address 127.0.0.1:55388)
0808 14:28:48.141520 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:48.141535 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:48.141624 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:48.142298 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:48.142300 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:48.142315 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:28:48.142413 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.142422 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.143873 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.143876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.145067 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.145078 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.146271 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.146274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.146396 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.146419 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 14:28:48.146491 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 14:28:48.146548 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"mutex_wait_us":70,"thread_start_us":89,"threads_started":1}
I20260808 14:28:48.150305 9757 tablet_service.cc:1467] Processing CreateTablet for tablet 85d6cf0807a842a0928c1bf5983e5dc7 (DEFAULT_TABLE table=test-table [id=1f0c085dbb4541b68c496cef87348b1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:28:48.150305 9611 tablet_service.cc:1467] Processing CreateTablet for tablet 85d6cf0807a842a0928c1bf5983e5dc7 (DEFAULT_TABLE table=test-table [id=1f0c085dbb4541b68c496cef87348b1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:28:48.151567 9757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85d6cf0807a842a0928c1bf5983e5dc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:48.151567 9611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85d6cf0807a842a0928c1bf5983e5dc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:48.152882 9902 tablet_service.cc:1467] Processing CreateTablet for tablet 85d6cf0807a842a0928c1bf5983e5dc7 (DEFAULT_TABLE table=test-table [id=1f0c085dbb4541b68c496cef87348b1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:28:48.153757 9902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85d6cf0807a842a0928c1bf5983e5dc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:48.159361 9995 tablet_bootstrap.cc:492] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc: Bootstrap starting.
I20260808 14:28:48.160475 9997 tablet_bootstrap.cc:492] T 85d6cf0807a842a0928c1bf5983e5dc7 P ad9701502ca34fa0b897099cca03d11d: Bootstrap starting.
I20260808 14:28:48.161563 9996 tablet_bootstrap.cc:492] T 85d6cf0807a842a0928c1bf5983e5dc7 P a50fe1e1869147b7adacc0b0a821006a: Bootstrap starting.
I20260808 14:28:48.162482 9995 tablet_bootstrap.cc:654] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:48.162608 9997 tablet_bootstrap.cc:654] T 85d6cf0807a842a0928c1bf5983e5dc7 P ad9701502ca34fa0b897099cca03d11d: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:48.163403 9997 log.cc:824] T 85d6cf0807a842a0928c1bf5983e5dc7 P ad9701502ca34fa0b897099cca03d11d: Log is configured to *not* fsync() on all Append() calls
I20260808 14:28:48.163411 9995 log.cc:824] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc: Log is configured to *not* fsync() on all Append() calls
I20260808 14:28:48.163720 9996 tablet_bootstrap.cc:654] T 85d6cf0807a842a0928c1bf5983e5dc7 P a50fe1e1869147b7adacc0b0a821006a: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:48.164491 9996 log.cc:824] T 85d6cf0807a842a0928c1bf5983e5dc7 P a50fe1e1869147b7adacc0b0a821006a: Log is configured to *not* fsync() on all Append() calls
I20260808 14:28:48.165400 9995 tablet_bootstrap.cc:492] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc: No bootstrap required, opened a new log
I20260808 14:28:48.165618 9995 ts_tablet_manager.cc:1397] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:28:48.166224 9996 tablet_bootstrap.cc:492] T 85d6cf0807a842a0928c1bf5983e5dc7 P a50fe1e1869147b7adacc0b0a821006a: No bootstrap required, opened a new log
I20260808 14:28:48.166410 9996 ts_tablet_manager.cc:1397] T 85d6cf0807a842a0928c1bf5983e5dc7 P a50fe1e1869147b7adacc0b0a821006a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 14:28:48.169512 9995 raft_consensus.cc:348] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } } peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } }
I20260808 14:28:48.169507 9996 raft_consensus.cc:348] T 85d6cf0807a842a0928c1bf5983e5dc7 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } } peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } }
I20260808 14:28:48.169778 9995 raft_consensus.cc:374] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:48.169778 9996 raft_consensus.cc:374] T 85d6cf0807a842a0928c1bf5983e5dc7 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:48.169832 9995 raft_consensus.cc:729] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4882374507a4151ba119338ce4daafc, State: Initialized, Role: FOLLOWER
I20260808 14:28:48.169835 9996 raft_consensus.cc:729] T 85d6cf0807a842a0928c1bf5983e5dc7 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a50fe1e1869147b7adacc0b0a821006a, State: Initialized, Role: FOLLOWER
I20260808 14:28:48.170145 9997 tablet_bootstrap.cc:492] T 85d6cf0807a842a0928c1bf5983e5dc7 P ad9701502ca34fa0b897099cca03d11d: No bootstrap required, opened a new log
I20260808 14:28:48.170325 9997 ts_tablet_manager.cc:1397] T 85d6cf0807a842a0928c1bf5983e5dc7 P ad9701502ca34fa0b897099cca03d11d: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260808 14:28:48.170279 9996 consensus_queue.cc:260] T 85d6cf0807a842a0928c1bf5983e5dc7 P a50fe1e1869147b7adacc0b0a821006a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } } peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } }
I20260808 14:28:48.170287 9995 consensus_queue.cc:260] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } } peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } }
I20260808 14:28:48.171097 9825 heartbeater.cc:503] Master 127.7.125.126:43091 was elected leader, sending a full tablet report...
I20260808 14:28:48.171445 9995 ts_tablet_manager.cc:1428] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260808 14:28:48.171801 9679 heartbeater.cc:503] Master 127.7.125.126:43091 was elected leader, sending a full tablet report...
I20260808 14:28:48.172398 9996 ts_tablet_manager.cc:1428] T 85d6cf0807a842a0928c1bf5983e5dc7 P a50fe1e1869147b7adacc0b0a821006a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 14:28:48.173907 9997 raft_consensus.cc:348] T 85d6cf0807a842a0928c1bf5983e5dc7 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } } peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } }
I20260808 14:28:48.174166 9997 raft_consensus.cc:374] T 85d6cf0807a842a0928c1bf5983e5dc7 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:48.174226 9997 raft_consensus.cc:729] T 85d6cf0807a842a0928c1bf5983e5dc7 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9701502ca34fa0b897099cca03d11d, State: Initialized, Role: FOLLOWER
I20260808 14:28:48.174660 9997 consensus_queue.cc:260] T 85d6cf0807a842a0928c1bf5983e5dc7 P ad9701502ca34fa0b897099cca03d11d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } } peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } }
I20260808 14:28:48.175377 9971 heartbeater.cc:503] Master 127.7.125.126:43091 was elected leader, sending a full tablet report...
I20260808 14:28:48.175731 9997 ts_tablet_manager.cc:1428] T 85d6cf0807a842a0928c1bf5983e5dc7 P ad9701502ca34fa0b897099cca03d11d: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260808 14:28:48.242749 10002 raft_consensus.cc:482] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:28:48.243016 10002 raft_consensus.cc:504] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } } peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } }
I20260808 14:28:48.244294 10002 leader_election.cc:288] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a50fe1e1869147b7adacc0b0a821006a (127.7.125.66:43119), ad9701502ca34fa0b897099cca03d11d (127.7.125.67:39819)
I20260808 14:28:48.248776 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.245189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:37295 (local address 127.7.125.67:39819)
0808 14:28:48.245341 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:48.245345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:48.245444 (+ 99us) server_negotiation.cc:407] Connection header received
0808 14:28:48.245592 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:48.245595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:48.245640 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:48.245721 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:28:48.245988 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.246562 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.247783 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.248193 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.248330 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.248387 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:48.248558 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 14:28:48.248635 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 14:28:48.248823 10005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.245046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:39819 (local address 127.7.125.65:37295)
0808 14:28:48.245367 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:48.245382 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:48.245487 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:48.245740 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:48.245742 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:48.245756 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:28:48.245848 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.245874 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.246677 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.246680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.247656 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.247665 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.248324 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.248328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.248419 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.248453 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 14:28:48.248521 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 14:28:48.248609 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":57,"thread_start_us":121,"threads_started":1}
I20260808 14:28:48.248809 9990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.244726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:44639 (local address 127.7.125.66:43119)
0808 14:28:48.244872 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:48.244877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:48.245176 (+ 299us) server_negotiation.cc:407] Connection header received
0808 14:28:48.245381 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:48.245385 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:48.245459 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:48.245538 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:28:48.245925 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.246489 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.247830 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.248213 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.248348 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.248411 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:48.248621 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 14:28:48.248707 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 14:28:48.248838 10004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.244604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:43119 (local address 127.7.125.65:44639)
0808 14:28:48.245036 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:48.245069 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:48.245209 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:48.245586 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:48.245589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:48.245609 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:28:48.245796 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.245807 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.246606 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.246612 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.247661 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.247670 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.248352 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.248356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.248463 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.248489 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 14:28:48.248586 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 14:28:48.248650 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":89,"thread_start_us":133,"threads_started":1}
I20260808 14:28:48.249569 9922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d6cf0807a842a0928c1bf5983e5dc7" candidate_uuid: "e4882374507a4151ba119338ce4daafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9701502ca34fa0b897099cca03d11d" is_pre_election: true
I20260808 14:28:48.249706 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d6cf0807a842a0928c1bf5983e5dc7" candidate_uuid: "e4882374507a4151ba119338ce4daafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a50fe1e1869147b7adacc0b0a821006a" is_pre_election: true
I20260808 14:28:48.249917 9922 raft_consensus.cc:2454] T 85d6cf0807a842a0928c1bf5983e5dc7 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4882374507a4151ba119338ce4daafc in term 0.
I20260808 14:28:48.249948 9777 raft_consensus.cc:2454] T 85d6cf0807a842a0928c1bf5983e5dc7 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4882374507a4151ba119338ce4daafc in term 0.
I20260808 14:28:48.250449 9557 leader_election.cc:302] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a50fe1e1869147b7adacc0b0a821006a, e4882374507a4151ba119338ce4daafc; no voters:
I20260808 14:28:48.250739 10002 raft_consensus.cc:2781] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:28:48.250840 10002 raft_consensus.cc:482] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:28:48.250926 10002 raft_consensus.cc:3037] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:48.252027 10002 raft_consensus.cc:504] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } } peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } }
I20260808 14:28:48.252382 10002 leader_election.cc:288] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc [CANDIDATE]: Term 1 election: Requested vote from peers a50fe1e1869147b7adacc0b0a821006a (127.7.125.66:43119), ad9701502ca34fa0b897099cca03d11d (127.7.125.67:39819)
I20260808 14:28:48.252748 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d6cf0807a842a0928c1bf5983e5dc7" candidate_uuid: "e4882374507a4151ba119338ce4daafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a50fe1e1869147b7adacc0b0a821006a"
I20260808 14:28:48.252822 9922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d6cf0807a842a0928c1bf5983e5dc7" candidate_uuid: "e4882374507a4151ba119338ce4daafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9701502ca34fa0b897099cca03d11d"
I20260808 14:28:48.252911 9777 raft_consensus.cc:3037] T 85d6cf0807a842a0928c1bf5983e5dc7 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:48.253005 9922 raft_consensus.cc:3037] T 85d6cf0807a842a0928c1bf5983e5dc7 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:48.254297 9922 raft_consensus.cc:2454] T 85d6cf0807a842a0928c1bf5983e5dc7 P ad9701502ca34fa0b897099cca03d11d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4882374507a4151ba119338ce4daafc in term 1.
I20260808 14:28:48.254307 9777 raft_consensus.cc:2454] T 85d6cf0807a842a0928c1bf5983e5dc7 P a50fe1e1869147b7adacc0b0a821006a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4882374507a4151ba119338ce4daafc in term 1.
I20260808 14:28:48.254591 9557 leader_election.cc:302] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a50fe1e1869147b7adacc0b0a821006a, e4882374507a4151ba119338ce4daafc; no voters:
I20260808 14:28:48.254760 10002 raft_consensus.cc:2781] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:28:48.255069 10002 raft_consensus.cc:686] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc [term 1 LEADER]: Becoming Leader. State: Replica: e4882374507a4151ba119338ce4daafc, State: Running, Role: LEADER
I20260808 14:28:48.255411 10002 consensus_queue.cc:237] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } } peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } }
I20260808 14:28:48.258644 9465 catalog_manager.cc:5495] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc reported cstate change: term changed from 0 to 1, leader changed from <none> to e4882374507a4151ba119338ce4daafc (127.7.125.65). New cstate: current_term: 1 leader_uuid: "e4882374507a4151ba119338ce4daafc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } health_report { overall_health: HEALTHY } } }
Aug 08 14:28:48 dist-test-slave-w206 krb5kdc[9418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199327, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:48.317648 10015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.308807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43091 (local address 127.0.0.1:37020)
0808 14:28:48.309127 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:48.309139 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:48.309270 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:48.309533 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:48.309536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:48.309956 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:48.310050 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.310056 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.310795 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.310801 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.311369 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.311377 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.311812 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.311815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.311893 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.312502 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:48.312515 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:48.314107 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:48.315938 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:48.315949 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:48.315955 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:48.316257 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:48.316557 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:48.316560 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:48.316562 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:48.316641 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:48.317039 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:48.317048 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:48.317159 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 14:28:48.317273 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 14:28:48.317446 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":107,"threads_started":1}
I20260808 14:28:48.317648 9517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.308858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37020 (local address 127.7.125.126:43091)
0808 14:28:48.309022 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:48.309027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:48.309217 (+ 190us) server_negotiation.cc:407] Connection header received
0808 14:28:48.309370 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:48.309375 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:48.309427 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:48.309512 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:48.310170 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.310689 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.311495 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.311701 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.311809 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.314240 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:48.314244 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:48.314246 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:48.314269 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:48.315804 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:48.316359 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:48.316364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:48.316366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:48.316449 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:48.316733 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:48.316737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:48.316738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:48.316917 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:48.317040 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:48.317329 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 14:28:48.317524 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":53}
I20260808 14:28:48.321660 9465 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:37020:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 14:28:48.322319 9465 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:28:48.327216 9757 tablet_service.cc:1467] Processing CreateTablet for tablet f63e619bbf1447f388b6e63e06ed7211 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f3700a53f5c9415986c4786692d9199f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 14:28:48.327193 9611 tablet_service.cc:1467] Processing CreateTablet for tablet f63e619bbf1447f388b6e63e06ed7211 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f3700a53f5c9415986c4786692d9199f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 14:28:48.327199 9902 tablet_service.cc:1467] Processing CreateTablet for tablet f63e619bbf1447f388b6e63e06ed7211 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f3700a53f5c9415986c4786692d9199f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 14:28:48.327903 9902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f63e619bbf1447f388b6e63e06ed7211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:48.327903 9611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f63e619bbf1447f388b6e63e06ed7211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:48.327917 9757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f63e619bbf1447f388b6e63e06ed7211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:48.330452 9996 tablet_bootstrap.cc:492] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a: Bootstrap starting.
I20260808 14:28:48.330796 9997 tablet_bootstrap.cc:492] T f63e619bbf1447f388b6e63e06ed7211 P ad9701502ca34fa0b897099cca03d11d: Bootstrap starting.
I20260808 14:28:48.331101 9995 tablet_bootstrap.cc:492] T f63e619bbf1447f388b6e63e06ed7211 P e4882374507a4151ba119338ce4daafc: Bootstrap starting.
I20260808 14:28:48.331519 9996 tablet_bootstrap.cc:654] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:48.331804 9997 tablet_bootstrap.cc:654] T f63e619bbf1447f388b6e63e06ed7211 P ad9701502ca34fa0b897099cca03d11d: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:48.332166 9995 tablet_bootstrap.cc:654] T f63e619bbf1447f388b6e63e06ed7211 P e4882374507a4151ba119338ce4daafc: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:48.332850 9996 tablet_bootstrap.cc:492] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a: No bootstrap required, opened a new log
I20260808 14:28:48.332953 9996 ts_tablet_manager.cc:1397] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:28:48.333125 9997 tablet_bootstrap.cc:492] T f63e619bbf1447f388b6e63e06ed7211 P ad9701502ca34fa0b897099cca03d11d: No bootstrap required, opened a new log
I20260808 14:28:48.333238 9997 ts_tablet_manager.cc:1397] T f63e619bbf1447f388b6e63e06ed7211 P ad9701502ca34fa0b897099cca03d11d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 14:28:48.333495 9996 raft_consensus.cc:348] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:48.333657 9996 raft_consensus.cc:374] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:48.333709 9996 raft_consensus.cc:729] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a50fe1e1869147b7adacc0b0a821006a, State: Initialized, Role: FOLLOWER
I20260808 14:28:48.333777 9997 raft_consensus.cc:348] T f63e619bbf1447f388b6e63e06ed7211 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:48.333847 9996 consensus_queue.cc:260] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:48.333905 9997 raft_consensus.cc:374] T f63e619bbf1447f388b6e63e06ed7211 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:48.333952 9997 raft_consensus.cc:729] T f63e619bbf1447f388b6e63e06ed7211 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9701502ca34fa0b897099cca03d11d, State: Initialized, Role: FOLLOWER
I20260808 14:28:48.333962 9995 tablet_bootstrap.cc:492] T f63e619bbf1447f388b6e63e06ed7211 P e4882374507a4151ba119338ce4daafc: No bootstrap required, opened a new log
I20260808 14:28:48.334034 9995 ts_tablet_manager.cc:1397] T f63e619bbf1447f388b6e63e06ed7211 P e4882374507a4151ba119338ce4daafc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:28:48.334131 9996 ts_tablet_manager.cc:1428] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 14:28:48.334077 9997 consensus_queue.cc:260] T f63e619bbf1447f388b6e63e06ed7211 P ad9701502ca34fa0b897099cca03d11d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:48.334362 9997 ts_tablet_manager.cc:1428] T f63e619bbf1447f388b6e63e06ed7211 P ad9701502ca34fa0b897099cca03d11d: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 14:28:48.334357 10001 tablet_replica.cc:439] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } } }
I20260808 14:28:48.334514 10001 tablet_replica.cc:442] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:28:48.334517 9995 raft_consensus.cc:348] T f63e619bbf1447f388b6e63e06ed7211 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:48.334617 9995 raft_consensus.cc:374] T f63e619bbf1447f388b6e63e06ed7211 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:48.334657 9995 raft_consensus.cc:729] T f63e619bbf1447f388b6e63e06ed7211 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4882374507a4151ba119338ce4daafc, State: Initialized, Role: FOLLOWER
I20260808 14:28:48.334596 10003 tablet_replica.cc:439] T f63e619bbf1447f388b6e63e06ed7211 P ad9701502ca34fa0b897099cca03d11d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } } }
I20260808 14:28:48.334712 10003 tablet_replica.cc:442] T f63e619bbf1447f388b6e63e06ed7211 P ad9701502ca34fa0b897099cca03d11d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:28:48.334784 9995 consensus_queue.cc:260] T f63e619bbf1447f388b6e63e06ed7211 P e4882374507a4151ba119338ce4daafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:48.335099 10002 tablet_replica.cc:439] T f63e619bbf1447f388b6e63e06ed7211 P e4882374507a4151ba119338ce4daafc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } } }
I20260808 14:28:48.335230 10002 tablet_replica.cc:442] T f63e619bbf1447f388b6e63e06ed7211 P e4882374507a4151ba119338ce4daafc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:28:48.335796 9995 ts_tablet_manager.cc:1428] T f63e619bbf1447f388b6e63e06ed7211 P e4882374507a4151ba119338ce4daafc: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260808 14:28:48.335894 9972 tablet.cc:2369] T 85d6cf0807a842a0928c1bf5983e5dc7 P ad9701502ca34fa0b897099cca03d11d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:28:48.348968 9826 tablet.cc:2369] T 85d6cf0807a842a0928c1bf5983e5dc7 P a50fe1e1869147b7adacc0b0a821006a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:28:48.375437 9680 tablet.cc:2369] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:28:48.446132 10001 raft_consensus.cc:482] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:28:48.446283 10001 raft_consensus.cc:504] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:48.447458 10001 leader_election.cc:288] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4882374507a4151ba119338ce4daafc (127.7.125.65:35287), ad9701502ca34fa0b897099cca03d11d (127.7.125.67:39819)
I20260808 14:28:48.451465 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.447867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:37519 (local address 127.7.125.67:39819)
0808 14:28:48.447998 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:48.448005 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:48.448239 (+ 234us) server_negotiation.cc:407] Connection header received
0808 14:28:48.448379 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:48.448382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:48.448432 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:48.448512 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:28:48.448797 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.449273 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.450359 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.450751 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.450874 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.450936 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:48.451241 (+ 305us) server_negotiation.cc:299] Negotiation successful
0808 14:28:48.451323 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 14:28:48.451493 9992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.447783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:44275 (local address 127.7.125.65:35287)
0808 14:28:48.447931 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:48.447936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:48.448110 (+ 174us) server_negotiation.cc:407] Connection header received
0808 14:28:48.448225 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:48.448229 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:48.448283 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:48.448363 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:28:48.448686 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.449275 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.450386 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.450765 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.450896 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.450949 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:48.451268 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 14:28:48.451357 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 14:28:48.451534 10019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.447667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:35287 (local address 127.7.125.66:44275)
0808 14:28:48.447993 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:48.448009 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:48.448126 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:48.448395 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:48.448398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:48.448413 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:28:48.448559 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.448566 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.449428 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.449431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.450230 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.450238 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.450910 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.450914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.451080 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.451098 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 14:28:48.451231 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 14:28:48.451337 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":48,"thread_start_us":79,"threads_started":1}
I20260808 14:28:48.451725 10020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.447764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:39819 (local address 127.7.125.66:37519)
0808 14:28:48.448120 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:48.448135 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:48.448251 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:48.448534 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:48.448536 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:48.448550 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:28:48.448667 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.448674 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.449381 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.449385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.450225 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.450233 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.450872 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.450876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.451006 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.451065 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 14:28:48.451214 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 14:28:48.451365 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":52,"thread_start_us":85,"threads_started":1}
I20260808 14:28:48.452124 9631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f63e619bbf1447f388b6e63e06ed7211" candidate_uuid: "a50fe1e1869147b7adacc0b0a821006a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4882374507a4151ba119338ce4daafc" is_pre_election: true
I20260808 14:28:48.452328 9922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f63e619bbf1447f388b6e63e06ed7211" candidate_uuid: "a50fe1e1869147b7adacc0b0a821006a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9701502ca34fa0b897099cca03d11d" is_pre_election: true
I20260808 14:28:48.452420 9631 raft_consensus.cc:2454] T f63e619bbf1447f388b6e63e06ed7211 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a50fe1e1869147b7adacc0b0a821006a in term 0.
I20260808 14:28:48.452499 9922 raft_consensus.cc:2454] T f63e619bbf1447f388b6e63e06ed7211 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a50fe1e1869147b7adacc0b0a821006a in term 0.
I20260808 14:28:48.452883 9704 leader_election.cc:302] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a50fe1e1869147b7adacc0b0a821006a, ad9701502ca34fa0b897099cca03d11d; no voters:
I20260808 14:28:48.453183 10001 raft_consensus.cc:2781] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:28:48.453259 10001 raft_consensus.cc:482] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:28:48.453308 10001 raft_consensus.cc:3037] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:48.454421 10001 raft_consensus.cc:504] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:48.454805 10001 leader_election.cc:288] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a [CANDIDATE]: Term 1 election: Requested vote from peers e4882374507a4151ba119338ce4daafc (127.7.125.65:35287), ad9701502ca34fa0b897099cca03d11d (127.7.125.67:39819)
I20260808 14:28:48.455128 9631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f63e619bbf1447f388b6e63e06ed7211" candidate_uuid: "a50fe1e1869147b7adacc0b0a821006a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4882374507a4151ba119338ce4daafc"
I20260808 14:28:48.455245 9631 raft_consensus.cc:3037] T f63e619bbf1447f388b6e63e06ed7211 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:48.455255 9922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f63e619bbf1447f388b6e63e06ed7211" candidate_uuid: "a50fe1e1869147b7adacc0b0a821006a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9701502ca34fa0b897099cca03d11d"
I20260808 14:28:48.455394 9922 raft_consensus.cc:3037] T f63e619bbf1447f388b6e63e06ed7211 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:48.456229 9631 raft_consensus.cc:2454] T f63e619bbf1447f388b6e63e06ed7211 P e4882374507a4151ba119338ce4daafc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a50fe1e1869147b7adacc0b0a821006a in term 1.
I20260808 14:28:48.456346 9922 raft_consensus.cc:2454] T f63e619bbf1447f388b6e63e06ed7211 P ad9701502ca34fa0b897099cca03d11d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a50fe1e1869147b7adacc0b0a821006a in term 1.
I20260808 14:28:48.456620 9704 leader_election.cc:302] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a50fe1e1869147b7adacc0b0a821006a, ad9701502ca34fa0b897099cca03d11d; no voters:
I20260808 14:28:48.456815 10001 raft_consensus.cc:2781] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:28:48.457013 10001 raft_consensus.cc:686] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a [term 1 LEADER]: Becoming Leader. State: Replica: a50fe1e1869147b7adacc0b0a821006a, State: Running, Role: LEADER
I20260808 14:28:48.457373 10001 consensus_queue.cc:237] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:48.459400 10021 tablet_replica.cc:439] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a: TxnStatusTablet state changed. Reason: New leader a50fe1e1869147b7adacc0b0a821006a. Latest consensus state: current_term: 1 leader_uuid: "a50fe1e1869147b7adacc0b0a821006a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } } }
I20260808 14:28:48.459539 10021 tablet_replica.cc:442] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a: This TxnStatusTablet replica's current role is: LEADER
I20260808 14:28:48.459838 10023 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 14:28:48.460054 10023 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 14:28:48.460745 9465 catalog_manager.cc:5495] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a reported cstate change: term changed from 0 to 1, leader changed from <none> to a50fe1e1869147b7adacc0b0a821006a (127.7.125.66). New cstate: current_term: 1 leader_uuid: "a50fe1e1869147b7adacc0b0a821006a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } health_report { overall_health: UNKNOWN } } }
I20260808 14:28:48.494371 10015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.489938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:43119 (local address 127.0.0.1:46414)
0808 14:28:48.490138 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:48.490159 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:48.490297 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:48.490704 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:48.490707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:48.490714 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 14:28:48.490814 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.490819 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.491877 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.491880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.492474 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.492482 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.493052 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.493055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.493128 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.493198 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 14:28:48.494021 (+ 823us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 14:28:48.494036 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 14:28:48.494092 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 14:28:48.494196 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 14:28:48.494663 9990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.490119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46414 (local address 127.7.125.66:43119)
0808 14:28:48.490274 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:48.490278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:48.490296 (+ 18us) server_negotiation.cc:407] Connection header received
0808 14:28:48.490494 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:48.490497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:48.490563 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:48.490666 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:28:48.490928 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.491744 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.492588 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.492932 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.493035 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.493350 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:28:48.493906 (+ 556us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:28:48.494047 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:48.494112 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 14:28:48.494181 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 14:28:48.500692 9631 raft_consensus.cc:1261] T f63e619bbf1447f388b6e63e06ed7211 P e4882374507a4151ba119338ce4daafc [term 1 FOLLOWER]: Refusing update from remote peer a50fe1e1869147b7adacc0b0a821006a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:28:48.500766 9922 raft_consensus.cc:1261] T f63e619bbf1447f388b6e63e06ed7211 P ad9701502ca34fa0b897099cca03d11d [term 1 FOLLOWER]: Refusing update from remote peer a50fe1e1869147b7adacc0b0a821006a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:28:48.501368 10021 consensus_queue.cc:1035] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:28:48.501549 10001 consensus_queue.cc:1035] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:28:48.504710 10002 tablet_replica.cc:439] T f63e619bbf1447f388b6e63e06ed7211 P e4882374507a4151ba119338ce4daafc: TxnStatusTablet state changed. Reason: New leader a50fe1e1869147b7adacc0b0a821006a. Latest consensus state: current_term: 1 leader_uuid: "a50fe1e1869147b7adacc0b0a821006a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } } }
I20260808 14:28:48.504841 10002 tablet_replica.cc:442] T f63e619bbf1447f388b6e63e06ed7211 P e4882374507a4151ba119338ce4daafc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:28:48.505501 10003 tablet_replica.cc:439] T f63e619bbf1447f388b6e63e06ed7211 P ad9701502ca34fa0b897099cca03d11d: TxnStatusTablet state changed. Reason: New leader a50fe1e1869147b7adacc0b0a821006a. Latest consensus state: current_term: 1 leader_uuid: "a50fe1e1869147b7adacc0b0a821006a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } } }
I20260808 14:28:48.505632 10003 tablet_replica.cc:442] T f63e619bbf1447f388b6e63e06ed7211 P ad9701502ca34fa0b897099cca03d11d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:28:48.508327 10021 tablet_replica.cc:439] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a50fe1e1869147b7adacc0b0a821006a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } } }
I20260808 14:28:48.508441 10021 tablet_replica.cc:442] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a: This TxnStatusTablet replica's current role is: LEADER
I20260808 14:28:48.509617 10003 tablet_replica.cc:439] T f63e619bbf1447f388b6e63e06ed7211 P ad9701502ca34fa0b897099cca03d11d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a50fe1e1869147b7adacc0b0a821006a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } } }
I20260808 14:28:48.509737 10003 tablet_replica.cc:442] T f63e619bbf1447f388b6e63e06ed7211 P ad9701502ca34fa0b897099cca03d11d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:28:48.510030 10028 mvcc.cc:205] Tried to move back new op lower bound from 7316272449531887616 to 7316272449363873792. Current Snapshot: MvccSnapshot[applied={T|T < 7316272449531887616}]
I20260808 14:28:48.511390 10001 tablet_replica.cc:439] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a50fe1e1869147b7adacc0b0a821006a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } } }
I20260808 14:28:48.511497 10001 tablet_replica.cc:442] T f63e619bbf1447f388b6e63e06ed7211 P a50fe1e1869147b7adacc0b0a821006a: This TxnStatusTablet replica's current role is: LEADER
I20260808 14:28:48.512746 10002 tablet_replica.cc:439] T f63e619bbf1447f388b6e63e06ed7211 P e4882374507a4151ba119338ce4daafc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a50fe1e1869147b7adacc0b0a821006a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } } }
I20260808 14:28:48.512869 10002 tablet_replica.cc:442] T f63e619bbf1447f388b6e63e06ed7211 P e4882374507a4151ba119338ce4daafc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:28:48.513969 10027 mvcc.cc:205] Tried to move back new op lower bound from 7316272449531887616 to 7316272449363873792. Current Snapshot: MvccSnapshot[applied={T|T < 7316272449531887616}]
I20260808 14:28:48.520740 9992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.516979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40728 (local address 127.7.125.65:35287)
0808 14:28:48.517093 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:48.517096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:48.517140 (+ 44us) server_negotiation.cc:407] Connection header received
0808 14:28:48.517305 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:48.517307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:48.517357 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:48.517439 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:28:48.517691 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.518189 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.519405 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.519606 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.519683 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.519975 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:28:48.520359 (+ 384us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:28:48.520446 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:48.520550 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 14:28:48.520594 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 14:28:48.531740 10040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.528093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:43119 (local address 127.7.125.65:51163)
0808 14:28:48.528503 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:48.528515 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:48.528654 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:48.528923 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:48.528926 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:48.528934 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 14:28:48.529081 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.529087 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.529822 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.529827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.530449 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.530457 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.530902 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.530906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.531029 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.531086 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 14:28:48.531471 (+ 385us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 14:28:48.531477 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 14:28:48.531535 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 14:28:48.531602 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":127,"threads_started":1}
I20260808 14:28:48.531767 9990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.528194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:51163 (local address 127.7.125.66:43119)
0808 14:28:48.528348 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:48.528353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:48.528609 (+ 256us) server_negotiation.cc:407] Connection header received
0808 14:28:48.528776 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:48.528779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:48.528827 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:48.528911 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:28:48.529192 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.529702 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.530575 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.530791 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.530900 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.531270 (+ 370us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:28:48.531361 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:28:48.531458 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:48.531572 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 14:28:48.531631 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 14:28:48.538954 9922 raft_consensus.cc:1261] T 85d6cf0807a842a0928c1bf5983e5dc7 P ad9701502ca34fa0b897099cca03d11d [term 1 FOLLOWER]: Refusing update from remote peer e4882374507a4151ba119338ce4daafc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:28:48.539286 9777 raft_consensus.cc:1261] T 85d6cf0807a842a0928c1bf5983e5dc7 P a50fe1e1869147b7adacc0b0a821006a [term 1 FOLLOWER]: Refusing update from remote peer e4882374507a4151ba119338ce4daafc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:28:48.539877 10006 consensus_queue.cc:1035] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:28:48.540134 10002 consensus_queue.cc:1035] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:28:48.544901 10026 mvcc.cc:205] Tried to move back new op lower bound from 7316272449691254784 to 7316272448537157632. Current Snapshot: MvccSnapshot[applied={T|T < 7316272449691254784}]
I20260808 14:28:48.564857 9992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.561046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:56729 (local address 127.7.125.65:35287)
0808 14:28:48.561175 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:48.561180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:48.561402 (+ 222us) server_negotiation.cc:407] Connection header received
0808 14:28:48.561539 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:48.561543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:48.561598 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:48.561696 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:28:48.561954 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.562574 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.563743 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.563984 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.564091 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.564383 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:28:48.564462 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:28:48.564541 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:48.564642 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 14:28:48.564698 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:28:48.564870 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.560961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:35287 (local address 127.7.125.66:56729)
0808 14:28:48.561296 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:48.561320 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:48.561431 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:48.561726 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:48.561729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:48.561739 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 14:28:48.561858 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.561865 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.562768 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.562771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.563607 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.563618 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.564092 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.564096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.564181 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.564234 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 14:28:48.564548 (+ 314us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 14:28:48.564557 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 14:28:48.564621 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 14:28:48.564702 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":127,"threads_started":1}
I20260808 14:28:48.593500 9464 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:28:48.593739 9464 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:28:48.597013 9464 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f: Sending DeleteTablet for 3 replicas of tablet 85d6cf0807a842a0928c1bf5983e5dc7
I20260808 14:28:48.597968 9611 tablet_service.cc:1514] Processing DeleteTablet for tablet 85d6cf0807a842a0928c1bf5983e5dc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:28:48 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:40724
I20260808 14:28:48.598168 9902 tablet_service.cc:1514] Processing DeleteTablet for tablet 85d6cf0807a842a0928c1bf5983e5dc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:28:48 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:55388
I20260808 14:28:48.598480 10048 tablet_replica.cc:330] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc: stopping tablet replica
I20260808 14:28:48.598774 10048 raft_consensus.cc:2229] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:28:48.599179 10048 raft_consensus.cc:2258] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:28:48.600147 9757 tablet_service.cc:1514] Processing DeleteTablet for tablet 85d6cf0807a842a0928c1bf5983e5dc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:28:48 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:46398
I20260808 14:28:48.600808 10048 ts_tablet_manager.cc:1905] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:28:48.600790 9464 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37010:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:28:48.601284 9464 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:28:48.603991 10048 ts_tablet_manager.cc:1918] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:28:48.605540 10048 log.cc:1196] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-0/wal/wals/85d6cf0807a842a0928c1bf5983e5dc7
I20260808 14:28:48.605863 10048 ts_tablet_manager.cc:1939] T 85d6cf0807a842a0928c1bf5983e5dc7 P e4882374507a4151ba119338ce4daafc: Deleting consensus metadata
I20260808 14:28:48.607602 9902 tablet_service.cc:1467] Processing CreateTablet for tablet d8edb3c6381447b68e00e19862781e89 (DEFAULT_TABLE table=test-table [id=832f9fa1c3764a78880e250e9ec8971f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:28:48.607764 9611 tablet_service.cc:1467] Processing CreateTablet for tablet d8edb3c6381447b68e00e19862781e89 (DEFAULT_TABLE table=test-table [id=832f9fa1c3764a78880e250e9ec8971f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:28:48.607923 9902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8edb3c6381447b68e00e19862781e89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:48.608000 9611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8edb3c6381447b68e00e19862781e89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:48.610502 10050 tablet_replica.cc:330] T 85d6cf0807a842a0928c1bf5983e5dc7 P a50fe1e1869147b7adacc0b0a821006a: stopping tablet replica
I20260808 14:28:48.610821 10050 raft_consensus.cc:2229] T 85d6cf0807a842a0928c1bf5983e5dc7 P a50fe1e1869147b7adacc0b0a821006a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:28:48.610965 9995 tablet_bootstrap.cc:492] T d8edb3c6381447b68e00e19862781e89 P e4882374507a4151ba119338ce4daafc: Bootstrap starting.
I20260808 14:28:48.611017 10050 raft_consensus.cc:2258] T 85d6cf0807a842a0928c1bf5983e5dc7 P a50fe1e1869147b7adacc0b0a821006a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:28:48.611245 10049 tablet_replica.cc:330] T 85d6cf0807a842a0928c1bf5983e5dc7 P ad9701502ca34fa0b897099cca03d11d: stopping tablet replica
I20260808 14:28:48.611508 10049 raft_consensus.cc:2229] T 85d6cf0807a842a0928c1bf5983e5dc7 P ad9701502ca34fa0b897099cca03d11d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:28:48.611683 10049 raft_consensus.cc:2258] T 85d6cf0807a842a0928c1bf5983e5dc7 P ad9701502ca34fa0b897099cca03d11d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:28:48.612082 9995 tablet_bootstrap.cc:654] T d8edb3c6381447b68e00e19862781e89 P e4882374507a4151ba119338ce4daafc: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:48.613076 9453 catalog_manager.cc:4841] TS e4882374507a4151ba119338ce4daafc (127.7.125.65:35287): tablet 85d6cf0807a842a0928c1bf5983e5dc7 (table test-table [id=1f0c085dbb4541b68c496cef87348b1c]) successfully deleted
I20260808 14:28:48.610823 9757 tablet_service.cc:1467] Processing CreateTablet for tablet d8edb3c6381447b68e00e19862781e89 (DEFAULT_TABLE table=test-table [id=832f9fa1c3764a78880e250e9ec8971f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:28:48.613857 9757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8edb3c6381447b68e00e19862781e89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:48.614143 10049 ts_tablet_manager.cc:1905] T 85d6cf0807a842a0928c1bf5983e5dc7 P ad9701502ca34fa0b897099cca03d11d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:28:48.614925 10050 ts_tablet_manager.cc:1905] T 85d6cf0807a842a0928c1bf5983e5dc7 P a50fe1e1869147b7adacc0b0a821006a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:28:48.616902 9996 tablet_bootstrap.cc:492] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a: Bootstrap starting.
I20260808 14:28:48.618013 10049 ts_tablet_manager.cc:1918] T 85d6cf0807a842a0928c1bf5983e5dc7 P ad9701502ca34fa0b897099cca03d11d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:28:48.618053 9996 tablet_bootstrap.cc:654] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:48.618099 10049 log.cc:1196] T 85d6cf0807a842a0928c1bf5983e5dc7 P ad9701502ca34fa0b897099cca03d11d: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/85d6cf0807a842a0928c1bf5983e5dc7
I20260808 14:28:48.618412 10049 ts_tablet_manager.cc:1939] T 85d6cf0807a842a0928c1bf5983e5dc7 P ad9701502ca34fa0b897099cca03d11d: Deleting consensus metadata
I20260808 14:28:48.619612 9997 tablet_bootstrap.cc:492] T d8edb3c6381447b68e00e19862781e89 P ad9701502ca34fa0b897099cca03d11d: Bootstrap starting.
I20260808 14:28:48.619652 9450 catalog_manager.cc:4841] TS ad9701502ca34fa0b897099cca03d11d (127.7.125.67:39819): tablet 85d6cf0807a842a0928c1bf5983e5dc7 (table test-table [id=1f0c085dbb4541b68c496cef87348b1c]) successfully deleted
I20260808 14:28:48.620647 9997 tablet_bootstrap.cc:654] T d8edb3c6381447b68e00e19862781e89 P ad9701502ca34fa0b897099cca03d11d: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:48.621924 10050 ts_tablet_manager.cc:1918] T 85d6cf0807a842a0928c1bf5983e5dc7 P a50fe1e1869147b7adacc0b0a821006a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:28:48.624756 9995 tablet_bootstrap.cc:492] T d8edb3c6381447b68e00e19862781e89 P e4882374507a4151ba119338ce4daafc: No bootstrap required, opened a new log
I20260808 14:28:48.624904 9995 ts_tablet_manager.cc:1397] T d8edb3c6381447b68e00e19862781e89 P e4882374507a4151ba119338ce4daafc: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.000s
I20260808 14:28:48.625442 9995 raft_consensus.cc:348] T d8edb3c6381447b68e00e19862781e89 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:48.625571 9995 raft_consensus.cc:374] T d8edb3c6381447b68e00e19862781e89 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:48.625603 9995 raft_consensus.cc:729] T d8edb3c6381447b68e00e19862781e89 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4882374507a4151ba119338ce4daafc, State: Initialized, Role: FOLLOWER
I20260808 14:28:48.625741 9995 consensus_queue.cc:260] T d8edb3c6381447b68e00e19862781e89 P e4882374507a4151ba119338ce4daafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:48.626044 9995 ts_tablet_manager.cc:1428] T d8edb3c6381447b68e00e19862781e89 P e4882374507a4151ba119338ce4daafc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:28:48.628348 9997 tablet_bootstrap.cc:492] T d8edb3c6381447b68e00e19862781e89 P ad9701502ca34fa0b897099cca03d11d: No bootstrap required, opened a new log
I20260808 14:28:48.628453 9997 ts_tablet_manager.cc:1397] T d8edb3c6381447b68e00e19862781e89 P ad9701502ca34fa0b897099cca03d11d: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.001s
I20260808 14:28:48.628973 9997 raft_consensus.cc:348] T d8edb3c6381447b68e00e19862781e89 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:48.629089 9997 raft_consensus.cc:374] T d8edb3c6381447b68e00e19862781e89 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:48.629122 9997 raft_consensus.cc:729] T d8edb3c6381447b68e00e19862781e89 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9701502ca34fa0b897099cca03d11d, State: Initialized, Role: FOLLOWER
I20260808 14:28:48.629261 9997 consensus_queue.cc:260] T d8edb3c6381447b68e00e19862781e89 P ad9701502ca34fa0b897099cca03d11d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:48.629557 9997 ts_tablet_manager.cc:1428] T d8edb3c6381447b68e00e19862781e89 P ad9701502ca34fa0b897099cca03d11d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:28:48.630183 9996 tablet_bootstrap.cc:492] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a: No bootstrap required, opened a new log
I20260808 14:28:48.630272 9996 ts_tablet_manager.cc:1397] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20260808 14:28:48.630764 9996 raft_consensus.cc:348] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:48.630882 9996 raft_consensus.cc:374] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:48.630913 9996 raft_consensus.cc:729] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a50fe1e1869147b7adacc0b0a821006a, State: Initialized, Role: FOLLOWER
I20260808 14:28:48.631069 9996 consensus_queue.cc:260] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:48.631371 9996 ts_tablet_manager.cc:1428] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 14:28:48.632665 10050 log.cc:1196] T 85d6cf0807a842a0928c1bf5983e5dc7 P a50fe1e1869147b7adacc0b0a821006a: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/85d6cf0807a842a0928c1bf5983e5dc7
I20260808 14:28:48.633068 10050 ts_tablet_manager.cc:1939] T 85d6cf0807a842a0928c1bf5983e5dc7 P a50fe1e1869147b7adacc0b0a821006a: Deleting consensus metadata
I20260808 14:28:48.634366 9450 catalog_manager.cc:4841] TS a50fe1e1869147b7adacc0b0a821006a (127.7.125.66:43119): tablet 85d6cf0807a842a0928c1bf5983e5dc7 (table test-table [id=1f0c085dbb4541b68c496cef87348b1c]) successfully deleted
I20260808 14:28:48.690460 10034 raft_consensus.cc:482] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:28:48.690668 10034 raft_consensus.cc:504] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:48.691237 10034 leader_election.cc:288] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4882374507a4151ba119338ce4daafc (127.7.125.65:35287), ad9701502ca34fa0b897099cca03d11d (127.7.125.67:39819)
I20260808 14:28:48.691706 9922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8edb3c6381447b68e00e19862781e89" candidate_uuid: "a50fe1e1869147b7adacc0b0a821006a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9701502ca34fa0b897099cca03d11d" is_pre_election: true
I20260808 14:28:48.691694 9631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8edb3c6381447b68e00e19862781e89" candidate_uuid: "a50fe1e1869147b7adacc0b0a821006a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4882374507a4151ba119338ce4daafc" is_pre_election: true
I20260808 14:28:48.691877 9631 raft_consensus.cc:2454] T d8edb3c6381447b68e00e19862781e89 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a50fe1e1869147b7adacc0b0a821006a in term 0.
I20260808 14:28:48.691881 9922 raft_consensus.cc:2454] T d8edb3c6381447b68e00e19862781e89 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a50fe1e1869147b7adacc0b0a821006a in term 0.
I20260808 14:28:48.692215 9707 leader_election.cc:302] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a50fe1e1869147b7adacc0b0a821006a, e4882374507a4151ba119338ce4daafc; no voters:
I20260808 14:28:48.692427 10034 raft_consensus.cc:2781] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:28:48.692487 10034 raft_consensus.cc:482] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:28:48.692536 10034 raft_consensus.cc:3037] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:48.693729 10034 raft_consensus.cc:504] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:48.694183 10034 leader_election.cc:288] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a [CANDIDATE]: Term 1 election: Requested vote from peers e4882374507a4151ba119338ce4daafc (127.7.125.65:35287), ad9701502ca34fa0b897099cca03d11d (127.7.125.67:39819)
I20260808 14:28:48.694607 9631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8edb3c6381447b68e00e19862781e89" candidate_uuid: "a50fe1e1869147b7adacc0b0a821006a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4882374507a4151ba119338ce4daafc"
I20260808 14:28:48.694725 9631 raft_consensus.cc:3037] T d8edb3c6381447b68e00e19862781e89 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:48.694736 9922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8edb3c6381447b68e00e19862781e89" candidate_uuid: "a50fe1e1869147b7adacc0b0a821006a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9701502ca34fa0b897099cca03d11d"
I20260808 14:28:48.694892 9922 raft_consensus.cc:3037] T d8edb3c6381447b68e00e19862781e89 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:48.695951 9631 raft_consensus.cc:2454] T d8edb3c6381447b68e00e19862781e89 P e4882374507a4151ba119338ce4daafc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a50fe1e1869147b7adacc0b0a821006a in term 1.
I20260808 14:28:48.696050 9922 raft_consensus.cc:2454] T d8edb3c6381447b68e00e19862781e89 P ad9701502ca34fa0b897099cca03d11d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a50fe1e1869147b7adacc0b0a821006a in term 1.
I20260808 14:28:48.696317 9707 leader_election.cc:302] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a50fe1e1869147b7adacc0b0a821006a, e4882374507a4151ba119338ce4daafc; no voters:
I20260808 14:28:48.696470 10034 raft_consensus.cc:2781] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:28:48.696568 10034 raft_consensus.cc:686] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a [term 1 LEADER]: Becoming Leader. State: Replica: a50fe1e1869147b7adacc0b0a821006a, State: Running, Role: LEADER
I20260808 14:28:48.696708 10034 consensus_queue.cc:237] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:48.698505 9466 catalog_manager.cc:5495] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a reported cstate change: term changed from 0 to 1, leader changed from <none> to a50fe1e1869147b7adacc0b0a821006a (127.7.125.66). New cstate: current_term: 1 leader_uuid: "a50fe1e1869147b7adacc0b0a821006a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } health_report { overall_health: UNKNOWN } } }
I20260808 14:28:48.725585 9990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.722130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46420 (local address 127.7.125.66:43119)
0808 14:28:48.722243 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:48.722246 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:48.722259 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:28:48.722408 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:48.722411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:48.722457 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:48.722519 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:28:48.722806 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.723377 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.724517 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.724719 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.724789 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.725095 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:28:48.725162 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:28:48.725214 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:48.725354 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 14:28:48.725430 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 14:28:48.735872 9990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.732295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:39397 (local address 127.7.125.66:43119)
0808 14:28:48.732403 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:48.732406 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:48.732460 (+ 54us) server_negotiation.cc:407] Connection header received
0808 14:28:48.732585 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:48.732587 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:48.732625 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:48.732684 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:28:48.732959 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.733618 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.734758 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.735012 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.735104 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.735413 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:28:48.735483 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:28:48.735559 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:48.735694 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 14:28:48.735735 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 14:28:48.735980 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:48.732216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:43119 (local address 127.7.125.66:39397)
0808 14:28:48.732398 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:48.732414 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:48.732505 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:48.732730 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:48.732733 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:48.732742 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 14:28:48.732853 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.732860 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.733750 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.733754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.734634 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:48.734646 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:48.735137 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:48.735141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:48.735226 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:48.735259 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 14:28:48.735600 (+ 341us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 14:28:48.735608 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 14:28:48.735678 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 14:28:48.735769 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260808 14:28:48.741704 9631 raft_consensus.cc:1261] T d8edb3c6381447b68e00e19862781e89 P e4882374507a4151ba119338ce4daafc [term 1 FOLLOWER]: Refusing update from remote peer a50fe1e1869147b7adacc0b0a821006a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:28:48.741691 9922 raft_consensus.cc:1261] T d8edb3c6381447b68e00e19862781e89 P ad9701502ca34fa0b897099cca03d11d [term 1 FOLLOWER]: Refusing update from remote peer a50fe1e1869147b7adacc0b0a821006a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:28:48.742081 10056 consensus_queue.cc:1035] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:28:48.742242 10034 consensus_queue.cc:1035] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 14:28:48.758525 10026 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 14:28:50.314833 9466 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:28:50.315001 9466 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:28:50.317283 9466 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f: Sending DeleteTablet for 3 replicas of tablet d8edb3c6381447b68e00e19862781e89
I20260808 14:28:50.317863 9611 tablet_service.cc:1514] Processing DeleteTablet for tablet d8edb3c6381447b68e00e19862781e89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:28:50 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:40724
I20260808 14:28:50.317919 9902 tablet_service.cc:1514] Processing DeleteTablet for tablet d8edb3c6381447b68e00e19862781e89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:28:50 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:55388
I20260808 14:28:50.318150 9756 tablet_service.cc:1514] Processing DeleteTablet for tablet d8edb3c6381447b68e00e19862781e89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:28:50 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:46398
I20260808 14:28:50.318228 10079 tablet_replica.cc:330] T d8edb3c6381447b68e00e19862781e89 P e4882374507a4151ba119338ce4daafc: stopping tablet replica
I20260808 14:28:50.318359 10079 raft_consensus.cc:2229] T d8edb3c6381447b68e00e19862781e89 P e4882374507a4151ba119338ce4daafc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:28:50.318480 10079 raft_consensus.cc:2258] T d8edb3c6381447b68e00e19862781e89 P e4882374507a4151ba119338ce4daafc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:28:50.318508 10081 tablet_replica.cc:330] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a: stopping tablet replica
I20260808 14:28:50.318624 10081 raft_consensus.cc:2229] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:28:50.318946 10081 raft_consensus.cc:2258] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:28:50.319162 10079 ts_tablet_manager.cc:1905] T d8edb3c6381447b68e00e19862781e89 P e4882374507a4151ba119338ce4daafc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:28:50.319581 10080 tablet_replica.cc:330] T d8edb3c6381447b68e00e19862781e89 P ad9701502ca34fa0b897099cca03d11d: stopping tablet replica
I20260808 14:28:50.319741 10080 raft_consensus.cc:2229] T d8edb3c6381447b68e00e19862781e89 P ad9701502ca34fa0b897099cca03d11d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:28:50.319778 10081 ts_tablet_manager.cc:1905] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:28:50.319878 10080 raft_consensus.cc:2258] T d8edb3c6381447b68e00e19862781e89 P ad9701502ca34fa0b897099cca03d11d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:28:50.320613 10080 ts_tablet_manager.cc:1905] T d8edb3c6381447b68e00e19862781e89 P ad9701502ca34fa0b897099cca03d11d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:28:50.322081 10079 ts_tablet_manager.cc:1918] T d8edb3c6381447b68e00e19862781e89 P e4882374507a4151ba119338ce4daafc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 14:28:50.322165 10079 log.cc:1196] T d8edb3c6381447b68e00e19862781e89 P e4882374507a4151ba119338ce4daafc: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-0/wal/wals/d8edb3c6381447b68e00e19862781e89
I20260808 14:28:50.322455 10079 ts_tablet_manager.cc:1939] T d8edb3c6381447b68e00e19862781e89 P e4882374507a4151ba119338ce4daafc: Deleting consensus metadata
I20260808 14:28:50.323127 10081 ts_tablet_manager.cc:1918] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 14:28:50.323261 10081 log.cc:1196] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/d8edb3c6381447b68e00e19862781e89
I20260808 14:28:50.323271 9453 catalog_manager.cc:4841] TS e4882374507a4151ba119338ce4daafc (127.7.125.65:35287): tablet d8edb3c6381447b68e00e19862781e89 (table test-table [id=832f9fa1c3764a78880e250e9ec8971f]) successfully deleted
I20260808 14:28:50.323642 10081 ts_tablet_manager.cc:1939] T d8edb3c6381447b68e00e19862781e89 P a50fe1e1869147b7adacc0b0a821006a: Deleting consensus metadata
I20260808 14:28:50.323668 10080 ts_tablet_manager.cc:1918] T d8edb3c6381447b68e00e19862781e89 P ad9701502ca34fa0b897099cca03d11d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 14:28:50.323741 10080 log.cc:1196] T d8edb3c6381447b68e00e19862781e89 P ad9701502ca34fa0b897099cca03d11d: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/d8edb3c6381447b68e00e19862781e89
I20260808 14:28:50.324026 10080 ts_tablet_manager.cc:1939] T d8edb3c6381447b68e00e19862781e89 P ad9701502ca34fa0b897099cca03d11d: Deleting consensus metadata
I20260808 14:28:50.324309 9450 catalog_manager.cc:4841] TS a50fe1e1869147b7adacc0b0a821006a (127.7.125.66:43119): tablet d8edb3c6381447b68e00e19862781e89 (table test-table [id=832f9fa1c3764a78880e250e9ec8971f]) successfully deleted
I20260808 14:28:50.324811 9450 catalog_manager.cc:4841] TS ad9701502ca34fa0b897099cca03d11d (127.7.125.67:39819): tablet d8edb3c6381447b68e00e19862781e89 (table test-table [id=832f9fa1c3764a78880e250e9ec8971f]) successfully deleted
Aug 08 14:28:50 dist-test-slave-w206 krb5kdc[9418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199330, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 14:28:50 dist-test-slave-w206 krb5kdc[9418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199330, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:50.348628 10090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:50.339740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37024 (local address 127.7.125.126:43091)
0808 14:28:50.340051 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:50.340061 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:50.340147 (+ 86us) server_negotiation.cc:407] Connection header received
0808 14:28:50.340326 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:50.340329 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:50.340383 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:50.340482 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:50.341183 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:50.341803 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:50.342657 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:50.342895 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:50.343020 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:50.345356 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:50.345360 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:50.345362 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:50.345385 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:50.346930 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:50.347380 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:50.347382 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:50.347383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:50.347463 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:50.347716 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:50.347720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:50.347721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:50.347915 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:50.348063 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:50.348318 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 14:28:50.348450 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":105,"threads_started":1}
I20260808 14:28:50.351120 9466 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:28:50.351574 9466 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:28:50.356345 9611 tablet_service.cc:1467] Processing CreateTablet for tablet e0a8bfff0f49482285d1bab2f4d272b5 (DEFAULT_TABLE table=test-table [id=e48bfff97963457c804bcb792785db4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:28:50.356415 9756 tablet_service.cc:1467] Processing CreateTablet for tablet e0a8bfff0f49482285d1bab2f4d272b5 (DEFAULT_TABLE table=test-table [id=e48bfff97963457c804bcb792785db4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:28:50.356666 9611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0a8bfff0f49482285d1bab2f4d272b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:50.356681 9756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0a8bfff0f49482285d1bab2f4d272b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:50.356686 9902 tablet_service.cc:1467] Processing CreateTablet for tablet e0a8bfff0f49482285d1bab2f4d272b5 (DEFAULT_TABLE table=test-table [id=e48bfff97963457c804bcb792785db4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:28:50.356935 9902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0a8bfff0f49482285d1bab2f4d272b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:50.359403 10092 tablet_bootstrap.cc:492] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc: Bootstrap starting.
I20260808 14:28:50.359966 10093 tablet_bootstrap.cc:492] T e0a8bfff0f49482285d1bab2f4d272b5 P ad9701502ca34fa0b897099cca03d11d: Bootstrap starting.
I20260808 14:28:50.360476 10092 tablet_bootstrap.cc:654] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:50.361016 10093 tablet_bootstrap.cc:654] T e0a8bfff0f49482285d1bab2f4d272b5 P ad9701502ca34fa0b897099cca03d11d: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:50.361773 10092 tablet_bootstrap.cc:492] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc: No bootstrap required, opened a new log
I20260808 14:28:50.361876 10092 ts_tablet_manager.cc:1397] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:28:50.362337 10092 raft_consensus.cc:348] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:50.362461 10092 raft_consensus.cc:374] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:50.362496 10092 raft_consensus.cc:729] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4882374507a4151ba119338ce4daafc, State: Initialized, Role: FOLLOWER
I20260808 14:28:50.362627 10092 consensus_queue.cc:260] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:50.362784 10093 tablet_bootstrap.cc:492] T e0a8bfff0f49482285d1bab2f4d272b5 P ad9701502ca34fa0b897099cca03d11d: No bootstrap required, opened a new log
I20260808 14:28:50.362875 10093 ts_tablet_manager.cc:1397] T e0a8bfff0f49482285d1bab2f4d272b5 P ad9701502ca34fa0b897099cca03d11d: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260808 14:28:50.363068 10092 ts_tablet_manager.cc:1428] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 14:28:50.363376 10093 raft_consensus.cc:348] T e0a8bfff0f49482285d1bab2f4d272b5 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:50.363492 10093 raft_consensus.cc:374] T e0a8bfff0f49482285d1bab2f4d272b5 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:50.363536 10093 raft_consensus.cc:729] T e0a8bfff0f49482285d1bab2f4d272b5 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9701502ca34fa0b897099cca03d11d, State: Initialized, Role: FOLLOWER
I20260808 14:28:50.363667 10093 consensus_queue.cc:260] T e0a8bfff0f49482285d1bab2f4d272b5 P ad9701502ca34fa0b897099cca03d11d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:50.364060 10093 ts_tablet_manager.cc:1428] T e0a8bfff0f49482285d1bab2f4d272b5 P ad9701502ca34fa0b897099cca03d11d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 14:28:50.364912 10091 tablet_bootstrap.cc:492] T e0a8bfff0f49482285d1bab2f4d272b5 P a50fe1e1869147b7adacc0b0a821006a: Bootstrap starting.
I20260808 14:28:50.365923 10091 tablet_bootstrap.cc:654] T e0a8bfff0f49482285d1bab2f4d272b5 P a50fe1e1869147b7adacc0b0a821006a: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:50.367195 10091 tablet_bootstrap.cc:492] T e0a8bfff0f49482285d1bab2f4d272b5 P a50fe1e1869147b7adacc0b0a821006a: No bootstrap required, opened a new log
I20260808 14:28:50.367273 10091 ts_tablet_manager.cc:1397] T e0a8bfff0f49482285d1bab2f4d272b5 P a50fe1e1869147b7adacc0b0a821006a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 14:28:50.367599 10091 raft_consensus.cc:348] T e0a8bfff0f49482285d1bab2f4d272b5 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:50.367689 10091 raft_consensus.cc:374] T e0a8bfff0f49482285d1bab2f4d272b5 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:50.367710 10091 raft_consensus.cc:729] T e0a8bfff0f49482285d1bab2f4d272b5 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a50fe1e1869147b7adacc0b0a821006a, State: Initialized, Role: FOLLOWER
I20260808 14:28:50.367851 10091 consensus_queue.cc:260] T e0a8bfff0f49482285d1bab2f4d272b5 P a50fe1e1869147b7adacc0b0a821006a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:50.368084 10091 ts_tablet_manager.cc:1428] T e0a8bfff0f49482285d1bab2f4d272b5 P a50fe1e1869147b7adacc0b0a821006a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 14:28:50.481321 10096 raft_consensus.cc:482] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:28:50.481542 10096 raft_consensus.cc:504] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:50.482026 10096 leader_election.cc:288] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a50fe1e1869147b7adacc0b0a821006a (127.7.125.66:43119), ad9701502ca34fa0b897099cca03d11d (127.7.125.67:39819)
I20260808 14:28:50.482525 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0a8bfff0f49482285d1bab2f4d272b5" candidate_uuid: "e4882374507a4151ba119338ce4daafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a50fe1e1869147b7adacc0b0a821006a" is_pre_election: true
I20260808 14:28:50.482565 9922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0a8bfff0f49482285d1bab2f4d272b5" candidate_uuid: "e4882374507a4151ba119338ce4daafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9701502ca34fa0b897099cca03d11d" is_pre_election: true
I20260808 14:28:50.482697 9777 raft_consensus.cc:2454] T e0a8bfff0f49482285d1bab2f4d272b5 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4882374507a4151ba119338ce4daafc in term 0.
I20260808 14:28:50.482748 9922 raft_consensus.cc:2454] T e0a8bfff0f49482285d1bab2f4d272b5 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4882374507a4151ba119338ce4daafc in term 0.
I20260808 14:28:50.483063 9557 leader_election.cc:302] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a50fe1e1869147b7adacc0b0a821006a, e4882374507a4151ba119338ce4daafc; no voters:
I20260808 14:28:50.483253 10096 raft_consensus.cc:2781] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:28:50.483316 10096 raft_consensus.cc:482] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:28:50.483358 10096 raft_consensus.cc:3037] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:50.484387 10096 raft_consensus.cc:504] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:50.484723 10096 leader_election.cc:288] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc [CANDIDATE]: Term 1 election: Requested vote from peers a50fe1e1869147b7adacc0b0a821006a (127.7.125.66:43119), ad9701502ca34fa0b897099cca03d11d (127.7.125.67:39819)
I20260808 14:28:50.485122 9922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0a8bfff0f49482285d1bab2f4d272b5" candidate_uuid: "e4882374507a4151ba119338ce4daafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9701502ca34fa0b897099cca03d11d"
I20260808 14:28:50.485113 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0a8bfff0f49482285d1bab2f4d272b5" candidate_uuid: "e4882374507a4151ba119338ce4daafc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a50fe1e1869147b7adacc0b0a821006a"
I20260808 14:28:50.485241 9922 raft_consensus.cc:3037] T e0a8bfff0f49482285d1bab2f4d272b5 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:50.485263 9777 raft_consensus.cc:3037] T e0a8bfff0f49482285d1bab2f4d272b5 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:50.486394 9777 raft_consensus.cc:2454] T e0a8bfff0f49482285d1bab2f4d272b5 P a50fe1e1869147b7adacc0b0a821006a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4882374507a4151ba119338ce4daafc in term 1.
I20260808 14:28:50.486394 9922 raft_consensus.cc:2454] T e0a8bfff0f49482285d1bab2f4d272b5 P ad9701502ca34fa0b897099cca03d11d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4882374507a4151ba119338ce4daafc in term 1.
I20260808 14:28:50.486689 9557 leader_election.cc:302] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a50fe1e1869147b7adacc0b0a821006a, e4882374507a4151ba119338ce4daafc; no voters:
I20260808 14:28:50.486858 10096 raft_consensus.cc:2781] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:28:50.487114 10096 raft_consensus.cc:686] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc [term 1 LEADER]: Becoming Leader. State: Replica: e4882374507a4151ba119338ce4daafc, State: Running, Role: LEADER
I20260808 14:28:50.487286 10096 consensus_queue.cc:237] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:50.488955 9466 catalog_manager.cc:5495] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc reported cstate change: term changed from 0 to 1, leader changed from <none> to e4882374507a4151ba119338ce4daafc (127.7.125.65). New cstate: current_term: 1 leader_uuid: "e4882374507a4151ba119338ce4daafc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } health_report { overall_health: UNKNOWN } } }
I20260808 14:28:50.525591 10102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:50.522245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40740 (local address 127.7.125.65:35287)
0808 14:28:50.522584 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:50.522593 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:50.522613 (+ 20us) server_negotiation.cc:407] Connection header received
0808 14:28:50.522651 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:50.522653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:50.522693 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:50.522789 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:28:50.523093 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:50.523626 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:50.524509 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:50.524741 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:50.524853 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:50.525118 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:28:50.525193 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:28:50.525277 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:50.525369 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 14:28:50.525425 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":136,"threads_started":1}
I20260808 14:28:50.531716 9922 raft_consensus.cc:1261] T e0a8bfff0f49482285d1bab2f4d272b5 P ad9701502ca34fa0b897099cca03d11d [term 1 FOLLOWER]: Refusing update from remote peer e4882374507a4151ba119338ce4daafc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:28:50.531827 9777 raft_consensus.cc:1261] T e0a8bfff0f49482285d1bab2f4d272b5 P a50fe1e1869147b7adacc0b0a821006a [term 1 FOLLOWER]: Refusing update from remote peer e4882374507a4151ba119338ce4daafc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:28:50.532188 10099 consensus_queue.cc:1035] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:28:50.532363 10096 consensus_queue.cc:1035] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:28:50.561017 9466 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:28:50.561193 9466 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:28:50.563336 9466 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f: Sending DeleteTablet for 3 replicas of tablet e0a8bfff0f49482285d1bab2f4d272b5
I20260808 14:28:50.563867 9611 tablet_service.cc:1514] Processing DeleteTablet for tablet e0a8bfff0f49482285d1bab2f4d272b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:28:50 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:40724
I20260808 14:28:50.563876 9756 tablet_service.cc:1514] Processing DeleteTablet for tablet e0a8bfff0f49482285d1bab2f4d272b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:28:50 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:46398
I20260808 14:28:50.564072 10079 tablet_replica.cc:330] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc: stopping tablet replica
I20260808 14:28:50.564054 9902 tablet_service.cc:1514] Processing DeleteTablet for tablet e0a8bfff0f49482285d1bab2f4d272b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:28:50 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:55388
I20260808 14:28:50.564108 10081 tablet_replica.cc:330] T e0a8bfff0f49482285d1bab2f4d272b5 P a50fe1e1869147b7adacc0b0a821006a: stopping tablet replica
I20260808 14:28:50.564206 10081 raft_consensus.cc:2229] T e0a8bfff0f49482285d1bab2f4d272b5 P a50fe1e1869147b7adacc0b0a821006a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:28:50.564203 10079 raft_consensus.cc:2229] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:28:50.564210 10080 tablet_replica.cc:330] T e0a8bfff0f49482285d1bab2f4d272b5 P ad9701502ca34fa0b897099cca03d11d: stopping tablet replica
I20260808 14:28:50.564316 10081 raft_consensus.cc:2258] T e0a8bfff0f49482285d1bab2f4d272b5 P a50fe1e1869147b7adacc0b0a821006a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:28:50.564345 10080 raft_consensus.cc:2229] T e0a8bfff0f49482285d1bab2f4d272b5 P ad9701502ca34fa0b897099cca03d11d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:28:50.564388 10079 raft_consensus.cc:2258] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:28:50.564462 10080 raft_consensus.cc:2258] T e0a8bfff0f49482285d1bab2f4d272b5 P ad9701502ca34fa0b897099cca03d11d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:28:50.565219 10080 ts_tablet_manager.cc:1905] T e0a8bfff0f49482285d1bab2f4d272b5 P ad9701502ca34fa0b897099cca03d11d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:28:50.565243 10079 ts_tablet_manager.cc:1905] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:28:50.565380 9466 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:28:50.565775 9466 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:28:50.567628 10080 ts_tablet_manager.cc:1918] T e0a8bfff0f49482285d1bab2f4d272b5 P ad9701502ca34fa0b897099cca03d11d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:28:50.567626 10079 ts_tablet_manager.cc:1918] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:28:50.567692 10080 log.cc:1196] T e0a8bfff0f49482285d1bab2f4d272b5 P ad9701502ca34fa0b897099cca03d11d: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/e0a8bfff0f49482285d1bab2f4d272b5
I20260808 14:28:50.567725 10079 log.cc:1196] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-0/wal/wals/e0a8bfff0f49482285d1bab2f4d272b5
I20260808 14:28:50.567970 10080 ts_tablet_manager.cc:1939] T e0a8bfff0f49482285d1bab2f4d272b5 P ad9701502ca34fa0b897099cca03d11d: Deleting consensus metadata
I20260808 14:28:50.567987 10079 ts_tablet_manager.cc:1939] T e0a8bfff0f49482285d1bab2f4d272b5 P e4882374507a4151ba119338ce4daafc: Deleting consensus metadata
I20260808 14:28:50.568779 9453 catalog_manager.cc:4841] TS e4882374507a4151ba119338ce4daafc (127.7.125.65:35287): tablet e0a8bfff0f49482285d1bab2f4d272b5 (table test-table [id=e48bfff97963457c804bcb792785db4f]) successfully deleted
I20260808 14:28:50.568831 9450 catalog_manager.cc:4841] TS ad9701502ca34fa0b897099cca03d11d (127.7.125.67:39819): tablet e0a8bfff0f49482285d1bab2f4d272b5 (table test-table [id=e48bfff97963457c804bcb792785db4f]) successfully deleted
I20260808 14:28:50.569653 10081 ts_tablet_manager.cc:1905] T e0a8bfff0f49482285d1bab2f4d272b5 P a50fe1e1869147b7adacc0b0a821006a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:28:50.571664 9611 tablet_service.cc:1467] Processing CreateTablet for tablet 7197b967acaf46a59adbde25be8bab20 (DEFAULT_TABLE table=test-table [id=85bf3885dd4647a08076f48d053b45e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:28:50.571939 9611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7197b967acaf46a59adbde25be8bab20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:50.571988 9902 tablet_service.cc:1467] Processing CreateTablet for tablet 7197b967acaf46a59adbde25be8bab20 (DEFAULT_TABLE table=test-table [id=85bf3885dd4647a08076f48d053b45e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:28:50.572211 9902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7197b967acaf46a59adbde25be8bab20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:50.572397 10081 ts_tablet_manager.cc:1918] T e0a8bfff0f49482285d1bab2f4d272b5 P a50fe1e1869147b7adacc0b0a821006a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:28:50.572453 10081 log.cc:1196] T e0a8bfff0f49482285d1bab2f4d272b5 P a50fe1e1869147b7adacc0b0a821006a: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/e0a8bfff0f49482285d1bab2f4d272b5
I20260808 14:28:50.572641 10081 ts_tablet_manager.cc:1939] T e0a8bfff0f49482285d1bab2f4d272b5 P a50fe1e1869147b7adacc0b0a821006a: Deleting consensus metadata
I20260808 14:28:50.574218 10092 tablet_bootstrap.cc:492] T 7197b967acaf46a59adbde25be8bab20 P e4882374507a4151ba119338ce4daafc: Bootstrap starting.
I20260808 14:28:50.574515 10093 tablet_bootstrap.cc:492] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d: Bootstrap starting.
I20260808 14:28:50.575074 10092 tablet_bootstrap.cc:654] T 7197b967acaf46a59adbde25be8bab20 P e4882374507a4151ba119338ce4daafc: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:50.575420 10093 tablet_bootstrap.cc:654] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:50.576215 10092 tablet_bootstrap.cc:492] T 7197b967acaf46a59adbde25be8bab20 P e4882374507a4151ba119338ce4daafc: No bootstrap required, opened a new log
I20260808 14:28:50.576175 9450 catalog_manager.cc:4841] TS a50fe1e1869147b7adacc0b0a821006a (127.7.125.66:43119): tablet e0a8bfff0f49482285d1bab2f4d272b5 (table test-table [id=e48bfff97963457c804bcb792785db4f]) successfully deleted
I20260808 14:28:50.576325 10092 ts_tablet_manager.cc:1397] T 7197b967acaf46a59adbde25be8bab20 P e4882374507a4151ba119338ce4daafc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 14:28:50.576622 9756 tablet_service.cc:1467] Processing CreateTablet for tablet 7197b967acaf46a59adbde25be8bab20 (DEFAULT_TABLE table=test-table [id=85bf3885dd4647a08076f48d053b45e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:28:50.576659 10092 raft_consensus.cc:348] T 7197b967acaf46a59adbde25be8bab20 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:50.576789 10092 raft_consensus.cc:374] T 7197b967acaf46a59adbde25be8bab20 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:50.576830 10092 raft_consensus.cc:729] T 7197b967acaf46a59adbde25be8bab20 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4882374507a4151ba119338ce4daafc, State: Initialized, Role: FOLLOWER
I20260808 14:28:50.576869 9756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7197b967acaf46a59adbde25be8bab20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:50.576954 10092 consensus_queue.cc:260] T 7197b967acaf46a59adbde25be8bab20 P e4882374507a4151ba119338ce4daafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:50.577239 10092 ts_tablet_manager.cc:1428] T 7197b967acaf46a59adbde25be8bab20 P e4882374507a4151ba119338ce4daafc: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 14:28:50.579007 10091 tablet_bootstrap.cc:492] T 7197b967acaf46a59adbde25be8bab20 P a50fe1e1869147b7adacc0b0a821006a: Bootstrap starting.
I20260808 14:28:50.579428 10093 tablet_bootstrap.cc:492] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d: No bootstrap required, opened a new log
I20260808 14:28:50.579496 10093 ts_tablet_manager.cc:1397] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260808 14:28:50.579804 10093 raft_consensus.cc:348] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:50.579859 10091 tablet_bootstrap.cc:654] T 7197b967acaf46a59adbde25be8bab20 P a50fe1e1869147b7adacc0b0a821006a: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:50.579880 10093 raft_consensus.cc:374] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:50.579905 10093 raft_consensus.cc:729] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9701502ca34fa0b897099cca03d11d, State: Initialized, Role: FOLLOWER
I20260808 14:28:50.580058 10093 consensus_queue.cc:260] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:50.580353 10093 ts_tablet_manager.cc:1428] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 14:28:50.581028 10091 tablet_bootstrap.cc:492] T 7197b967acaf46a59adbde25be8bab20 P a50fe1e1869147b7adacc0b0a821006a: No bootstrap required, opened a new log
I20260808 14:28:50.581123 10091 ts_tablet_manager.cc:1397] T 7197b967acaf46a59adbde25be8bab20 P a50fe1e1869147b7adacc0b0a821006a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260808 14:28:50.581490 10091 raft_consensus.cc:348] T 7197b967acaf46a59adbde25be8bab20 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:50.581621 10091 raft_consensus.cc:374] T 7197b967acaf46a59adbde25be8bab20 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:50.581656 10091 raft_consensus.cc:729] T 7197b967acaf46a59adbde25be8bab20 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a50fe1e1869147b7adacc0b0a821006a, State: Initialized, Role: FOLLOWER
I20260808 14:28:50.581774 10091 consensus_queue.cc:260] T 7197b967acaf46a59adbde25be8bab20 P a50fe1e1869147b7adacc0b0a821006a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:50.582077 10091 ts_tablet_manager.cc:1428] T 7197b967acaf46a59adbde25be8bab20 P a50fe1e1869147b7adacc0b0a821006a: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260808 14:28:50.808308 10097 raft_consensus.cc:482] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:28:50.808460 10097 raft_consensus.cc:504] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:50.809602 10097 leader_election.cc:288] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4882374507a4151ba119338ce4daafc (127.7.125.65:35287), a50fe1e1869147b7adacc0b0a821006a (127.7.125.66:43119)
I20260808 14:28:50.813561 10102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:50.809941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:46081 (local address 127.7.125.65:35287)
0808 14:28:50.810077 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:50.810084 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:50.810277 (+ 193us) server_negotiation.cc:407] Connection header received
0808 14:28:50.810454 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:50.810456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:50.810502 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:50.810586 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:28:50.810857 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:50.811429 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:50.812524 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:50.812993 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:50.813121 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:50.813180 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:50.813371 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 14:28:50.813458 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 14:28:50.813582 10112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:50.809812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:35287 (local address 127.7.125.67:46081)
0808 14:28:50.810180 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:50.810193 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:50.810323 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:50.810599 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:50.810602 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:50.810618 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:28:50.810745 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:50.810751 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:50.811544 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:50.811548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:50.812400 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:50.812409 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:50.813121 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:50.813124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:50.813235 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:50.813265 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 14:28:50.813343 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 14:28:50.813418 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"spinlock_wait_cycles":11520,"thread_start_us":127,"threads_started":1}
I20260808 14:28:50.813687 10113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:50.809989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:35343 (local address 127.7.125.66:43119)
0808 14:28:50.810259 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:50.810278 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:50.810367 (+ 89us) server_negotiation.cc:407] Connection header received
0808 14:28:50.810523 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:50.810526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:50.810576 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:50.810668 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:28:50.810942 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:50.811521 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:50.812551 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:50.813026 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:50.813222 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:50.813271 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:50.813460 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 14:28:50.813536 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":81,"threads_started":1}
I20260808 14:28:50.813876 10111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:50.809830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:43119 (local address 127.7.125.67:35343)
0808 14:28:50.810259 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:50.810276 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:50.810420 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:50.810680 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:50.810683 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:50.810698 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:28:50.810832 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:50.810837 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:50.811650 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:50.811653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:50.812425 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:50.812433 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:50.813213 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:50.813217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:50.813335 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:50.813353 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 14:28:50.813441 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 14:28:50.813509 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":103,"threads_started":1}
I20260808 14:28:50.814381 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7197b967acaf46a59adbde25be8bab20" candidate_uuid: "ad9701502ca34fa0b897099cca03d11d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a50fe1e1869147b7adacc0b0a821006a" is_pre_election: true
I20260808 14:28:50.814383 9631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7197b967acaf46a59adbde25be8bab20" candidate_uuid: "ad9701502ca34fa0b897099cca03d11d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4882374507a4151ba119338ce4daafc" is_pre_election: true
I20260808 14:28:50.814536 9631 raft_consensus.cc:2454] T 7197b967acaf46a59adbde25be8bab20 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad9701502ca34fa0b897099cca03d11d in term 0.
I20260808 14:28:50.814536 9777 raft_consensus.cc:2454] T 7197b967acaf46a59adbde25be8bab20 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad9701502ca34fa0b897099cca03d11d in term 0.
I20260808 14:28:50.814908 9853 leader_election.cc:302] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a50fe1e1869147b7adacc0b0a821006a, ad9701502ca34fa0b897099cca03d11d, e4882374507a4151ba119338ce4daafc; no voters:
I20260808 14:28:50.815224 10097 raft_consensus.cc:2781] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:28:50.815300 10097 raft_consensus.cc:482] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:28:50.815356 10097 raft_consensus.cc:3037] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:50.816355 10097 raft_consensus.cc:504] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:50.816709 10097 leader_election.cc:288] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d [CANDIDATE]: Term 1 election: Requested vote from peers e4882374507a4151ba119338ce4daafc (127.7.125.65:35287), a50fe1e1869147b7adacc0b0a821006a (127.7.125.66:43119)
I20260808 14:28:50.817060 9777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7197b967acaf46a59adbde25be8bab20" candidate_uuid: "ad9701502ca34fa0b897099cca03d11d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a50fe1e1869147b7adacc0b0a821006a"
I20260808 14:28:50.817060 9631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7197b967acaf46a59adbde25be8bab20" candidate_uuid: "ad9701502ca34fa0b897099cca03d11d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4882374507a4151ba119338ce4daafc"
I20260808 14:28:50.817191 9631 raft_consensus.cc:3037] T 7197b967acaf46a59adbde25be8bab20 P e4882374507a4151ba119338ce4daafc [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:50.817191 9777 raft_consensus.cc:3037] T 7197b967acaf46a59adbde25be8bab20 P a50fe1e1869147b7adacc0b0a821006a [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:50.818298 9631 raft_consensus.cc:2454] T 7197b967acaf46a59adbde25be8bab20 P e4882374507a4151ba119338ce4daafc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad9701502ca34fa0b897099cca03d11d in term 1.
I20260808 14:28:50.818298 9777 raft_consensus.cc:2454] T 7197b967acaf46a59adbde25be8bab20 P a50fe1e1869147b7adacc0b0a821006a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad9701502ca34fa0b897099cca03d11d in term 1.
I20260808 14:28:50.818610 9850 leader_election.cc:302] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a50fe1e1869147b7adacc0b0a821006a, ad9701502ca34fa0b897099cca03d11d; no voters:
I20260808 14:28:50.818791 10097 raft_consensus.cc:2781] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:28:50.819092 10097 raft_consensus.cc:686] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d [term 1 LEADER]: Becoming Leader. State: Replica: ad9701502ca34fa0b897099cca03d11d, State: Running, Role: LEADER
I20260808 14:28:50.819381 10097 consensus_queue.cc:237] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } }
I20260808 14:28:50.821954 9466 catalog_manager.cc:5495] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d reported cstate change: term changed from 0 to 1, leader changed from <none> to ad9701502ca34fa0b897099cca03d11d (127.7.125.67). New cstate: current_term: 1 leader_uuid: "ad9701502ca34fa0b897099cca03d11d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9701502ca34fa0b897099cca03d11d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39819 } health_report { overall_health: HEALTHY } } }
I20260808 14:28:50.876487 10116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:50.872771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55396 (local address 127.7.125.67:39819)
0808 14:28:50.873107 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:50.873116 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:50.873128 (+ 12us) server_negotiation.cc:407] Connection header received
0808 14:28:50.873200 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:50.873202 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:50.873253 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:50.873356 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:28:50.873556 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:50.874198 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:50.875061 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:50.875298 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:50.875378 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:50.875615 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:28:50.876070 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:28:50.876159 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:50.876220 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 14:28:50.876266 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":146,"threads_started":1}
I20260808 14:28:50.886920 10113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:50.883500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:47997 (local address 127.7.125.66:43119)
0808 14:28:50.883641 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:50.883644 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:50.883660 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:28:50.883768 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:50.883771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:50.883818 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:50.883890 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:28:50.884108 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:50.884635 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:50.885507 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:50.885722 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:50.885807 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:50.886524 (+ 717us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:28:50.886591 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:28:50.886645 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:50.886751 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 14:28:50.886799 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 14:28:50.886945 10119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:50.883233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:43119 (local address 127.7.125.67:47997)
0808 14:28:50.883590 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:50.883603 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:50.883680 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:50.883909 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:50.883911 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:50.883917 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 14:28:50.884014 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:50.884018 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:50.884738 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:50.884740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:50.885374 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:50.885382 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:50.886264 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:50.886269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:50.886354 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:50.886398 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 14:28:50.886666 (+ 268us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 14:28:50.886672 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 14:28:50.886743 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 14:28:50.886811 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":112,"threads_started":1}
I20260808 14:28:50.893316 9631 raft_consensus.cc:1261] T 7197b967acaf46a59adbde25be8bab20 P e4882374507a4151ba119338ce4daafc [term 1 FOLLOWER]: Refusing update from remote peer ad9701502ca34fa0b897099cca03d11d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:28:50.893460 9777 raft_consensus.cc:1261] T 7197b967acaf46a59adbde25be8bab20 P a50fe1e1869147b7adacc0b0a821006a [term 1 FOLLOWER]: Refusing update from remote peer ad9701502ca34fa0b897099cca03d11d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:28:50.893784 10097 consensus_queue.cc:1035] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4882374507a4151ba119338ce4daafc" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:28:50.893977 10114 consensus_queue.cc:1035] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a50fe1e1869147b7adacc0b0a821006a" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 43119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:28:50.911521 10124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:50.908047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:39819 (local address 127.7.125.66:58913)
0808 14:28:50.908327 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:50.908347 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:50.908481 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:50.908714 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:50.908716 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:50.908722 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 14:28:50.908811 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:50.908815 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:50.909463 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:50.909466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:50.910176 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:50.910186 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:50.910769 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:50.910772 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:50.910863 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:50.910899 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 14:28:50.911247 (+ 348us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 14:28:50.911254 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 14:28:50.911329 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 14:28:50.911400 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":123,"threads_started":1}
I20260808 14:28:50.911525 10116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:50.908108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:58913 (local address 127.7.125.67:39819)
0808 14:28:50.908248 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:50.908254 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:50.908436 (+ 182us) server_negotiation.cc:407] Connection header received
0808 14:28:50.908571 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:50.908577 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:50.908629 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:50.908711 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:28:50.908874 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:50.909353 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:50.910313 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:50.910648 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:50.910772 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:50.911060 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:28:50.911150 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:28:50.911226 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:50.911345 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 14:28:50.911401 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 14:28:50.912246 10063 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 14:28:52.467311 9465 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:28:52.467514 9465 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:28:52.469906 9465 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b58df962d3fe423ab32a8136a5e1b79f: Sending DeleteTablet for 3 replicas of tablet 7197b967acaf46a59adbde25be8bab20
I20260808 14:28:52.470491 9756 tablet_service.cc:1514] Processing DeleteTablet for tablet 7197b967acaf46a59adbde25be8bab20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:28:52 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:46398
I20260808 14:28:52.470597 9611 tablet_service.cc:1514] Processing DeleteTablet for tablet 7197b967acaf46a59adbde25be8bab20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:28:52 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:40724
I20260808 14:28:52.470868 10148 tablet_replica.cc:330] T 7197b967acaf46a59adbde25be8bab20 P a50fe1e1869147b7adacc0b0a821006a: stopping tablet replica
I20260808 14:28:52.470909 9902 tablet_service.cc:1514] Processing DeleteTablet for tablet 7197b967acaf46a59adbde25be8bab20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:28:52 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:55388
I20260808 14:28:52.470991 10149 tablet_replica.cc:330] T 7197b967acaf46a59adbde25be8bab20 P e4882374507a4151ba119338ce4daafc: stopping tablet replica
I20260808 14:28:52.470963 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 9541
I20260808 14:28:52.471028 10148 raft_consensus.cc:2229] T 7197b967acaf46a59adbde25be8bab20 P a50fe1e1869147b7adacc0b0a821006a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:28:52.471153 10148 raft_consensus.cc:2258] T 7197b967acaf46a59adbde25be8bab20 P a50fe1e1869147b7adacc0b0a821006a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:28:52.472085 10148 ts_tablet_manager.cc:1905] T 7197b967acaf46a59adbde25be8bab20 P a50fe1e1869147b7adacc0b0a821006a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:28:52.474756 10148 ts_tablet_manager.cc:1918] T 7197b967acaf46a59adbde25be8bab20 P a50fe1e1869147b7adacc0b0a821006a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 14:28:52.474822 10148 log.cc:1196] T 7197b967acaf46a59adbde25be8bab20 P a50fe1e1869147b7adacc0b0a821006a: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/7197b967acaf46a59adbde25be8bab20
I20260808 14:28:52.475152 10148 ts_tablet_manager.cc:1939] T 7197b967acaf46a59adbde25be8bab20 P a50fe1e1869147b7adacc0b0a821006a: Deleting consensus metadata
I20260808 14:28:52.476008 9450 catalog_manager.cc:4841] TS a50fe1e1869147b7adacc0b0a821006a (127.7.125.66:43119): tablet 7197b967acaf46a59adbde25be8bab20 (table test-table [id=85bf3885dd4647a08076f48d053b45e1]) successfully deleted
I20260808 14:28:52.482012 10150 tablet_replica.cc:330] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d: stopping tablet replica
W20260808 14:28:52.482226 9453 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.125.65:35287 (error 108)
I20260808 14:28:52.482272 10150 raft_consensus.cc:2229] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:28:52.482621 10150 raft_consensus.cc:2258] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:28:52.483314 10150 ts_tablet_manager.cc:1905] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 14:28:52.483513 10152 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:28:52.483046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:35287 (local address 127.0.0.1:40744)
0808 14:28:52.483309 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:52.483400 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.65:35287: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":80,"threads_started":1}
I20260808 14:28:52.483645 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 9688
W20260808 14:28:52.483673 9453 catalog_manager.cc:4584] TS e4882374507a4151ba119338ce4daafc (127.7.125.65:35287): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7197b967acaf46a59adbde25be8bab20: Network error: Client connection negotiation failed: client connection to 127.7.125.65:35287: connect: Connection reset by peer (error 104)
I20260808 14:28:52.487962 10150 ts_tablet_manager.cc:1918] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 14:28:52.488039 10150 log.cc:1196] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/7197b967acaf46a59adbde25be8bab20
I20260808 14:28:52.488338 10150 ts_tablet_manager.cc:1939] T 7197b967acaf46a59adbde25be8bab20 P ad9701502ca34fa0b897099cca03d11d: Deleting consensus metadata
I20260808 14:28:52.489066 9450 catalog_manager.cc:4841] TS ad9701502ca34fa0b897099cca03d11d (127.7.125.67:39819): tablet 7197b967acaf46a59adbde25be8bab20 (table test-table [id=85bf3885dd4647a08076f48d053b45e1]) successfully deleted
I20260808 14:28:52.491852 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 9834
I20260808 14:28:52.499912 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 9432
2026-08-08T14:28:52Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6598 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:28:52 dist-test-slave-w206 krb5kdc[10155](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:28:52 dist-test-slave-w206 krb5kdc[10155](info): set up 2 sockets
Aug 08 14:28:52 dist-test-slave-w206 krb5kdc[10155](info): commencing operation
krb5kdc: starting...
W20260808 14:28:53.575227 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:28:53 dist-test-slave-w206 krb5kdc[10155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:28:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:28:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:28:53.707130 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:33445
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:33445
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:53.820783 10170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:53.821044 10170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:53.821085 10170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:53.825125 10170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:28:53.825189 10170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:53.825209 10170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:53.825275 10170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:28:53.825305 10170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:28:53.830719 10170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:33445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:33445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:53.831827 10170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:53.832958 10170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:28:53.841777 10175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:53.841847 10178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:53.841964 10170 server_base.cc:1034] running on GCE node
W20260808 14:28:53.841858 10176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:53.842644 10170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:53.843797 10170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:53.844957 10170 hybrid_clock.cc:648] HybridClock initialized: now 1786199333844932 us; error 40 us; skew 500 ppm
Aug 08 14:28:53 dist-test-slave-w206 krb5kdc[10155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199333, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:53.847826 10170 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:28:53.848770 10170 webserver.cc:460] Webserver started at http://127.7.125.126:43683/ using document root <none> and password file <none>
I20260808 14:28:53.849375 10170 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:53.849478 10170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:53.849720 10170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:53.851899 10170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "a04ae5f0bd5e4836883bcb79731216ef"
format_stamp: "Formatted at 2026-08-08 14:28:53 on dist-test-slave-w206"
server_key: "2397abafda312915295cca81fa4d86c4"
server_key_iv: "a6fc1f5d751ef49acdb9fda042f5a59f"
server_key_version: "encryptionkey@0"
I20260808 14:28:53.852471 10170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "a04ae5f0bd5e4836883bcb79731216ef"
format_stamp: "Formatted at 2026-08-08 14:28:53 on dist-test-slave-w206"
server_key: "2397abafda312915295cca81fa4d86c4"
server_key_iv: "a6fc1f5d751ef49acdb9fda042f5a59f"
server_key_version: "encryptionkey@0"
I20260808 14:28:53.856102 10170 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 14:28:53.858429 10185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:53.859627 10170 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:28:53.859768 10170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "a04ae5f0bd5e4836883bcb79731216ef"
format_stamp: "Formatted at 2026-08-08 14:28:53 on dist-test-slave-w206"
server_key: "2397abafda312915295cca81fa4d86c4"
server_key_iv: "a6fc1f5d751ef49acdb9fda042f5a59f"
server_key_version: "encryptionkey@0"
I20260808 14:28:53.859879 10170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:53.892773 10170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:53.896410 10170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:53.896687 10170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:53.904470 10170 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:33445
I20260808 14:28:53.904527 10237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:33445 every 8 connection(s)
I20260808 14:28:53.905519 10170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:28:53.908740 10238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:53.914353 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 10170
I20260808 14:28:53.914513 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:28:53.914845 7669 external_mini_cluster.cc:1428] Setting key 09bd8185f01b033f0376e0abd067acee
I20260808 14:28:53.915450 10238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef: Bootstrap starting.
I20260808 14:28:53.918514 10238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:53.919608 10238 log.cc:824] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef: Log is configured to *not* fsync() on all Append() calls
I20260808 14:28:53.922080 10238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef: No bootstrap required, opened a new log
Aug 08 14:28:53 dist-test-slave-w206 krb5kdc[10155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:53.927204 10238 raft_consensus.cc:348] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a04ae5f0bd5e4836883bcb79731216ef" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33445 } }
I20260808 14:28:53.927496 10238 raft_consensus.cc:374] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:53.927567 10238 raft_consensus.cc:729] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a04ae5f0bd5e4836883bcb79731216ef, State: Initialized, Role: FOLLOWER
I20260808 14:28:53.928112 10238 consensus_queue.cc:260] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a04ae5f0bd5e4836883bcb79731216ef" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33445 } }
I20260808 14:28:53.928267 10238 raft_consensus.cc:388] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:28:53.928372 10238 raft_consensus.cc:482] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:28:53.928511 10238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:53.929893 10238 raft_consensus.cc:504] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a04ae5f0bd5e4836883bcb79731216ef" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33445 } }
I20260808 14:28:53.930295 10238 leader_election.cc:302] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a04ae5f0bd5e4836883bcb79731216ef; no voters:
I20260808 14:28:53.930713 10238 leader_election.cc:288] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:28:53.930899 10244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:28:53.931180 10244 raft_consensus.cc:686] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef [term 1 LEADER]: Becoming Leader. State: Replica: a04ae5f0bd5e4836883bcb79731216ef, State: Running, Role: LEADER
I20260808 14:28:53.931531 10244 consensus_queue.cc:237] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a04ae5f0bd5e4836883bcb79731216ef" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33445 } }
I20260808 14:28:53.931938 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:53.916455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44386 (local address 127.7.125.126:33445)
0808 14:28:53.916962 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:53.916985 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:53.917034 (+ 49us) server_negotiation.cc:407] Connection header received
0808 14:28:53.917995 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:53.918020 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:53.918484 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:53.918926 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:53.919722 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:53.921336 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:53.922248 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:53.923087 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:53.923285 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:53.926418 (+ 3133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:53.926424 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:53.926438 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:53.926466 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:53.928502 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:53.929100 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:53.929105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:53.929109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:53.929228 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:53.929556 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:53.929558 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:53.929559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:53.929951 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:53.930155 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:53.930558 (+ 403us) server_negotiation.cc:299] Negotiation successful
0808 14:28:53.930892 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":131,"threads_started":1}
I20260808 14:28:53.932229 10238 sys_catalog.cc:564] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:28:53.932862 10244 sys_catalog.cc:455] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader a04ae5f0bd5e4836883bcb79731216ef. Latest consensus state: current_term: 1 leader_uuid: "a04ae5f0bd5e4836883bcb79731216ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a04ae5f0bd5e4836883bcb79731216ef" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33445 } } }
I20260808 14:28:53.932994 10244 sys_catalog.cc:458] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef [sys.catalog]: This master's current role is: LEADER
I20260808 14:28:53.933274 10246 sys_catalog.cc:455] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a04ae5f0bd5e4836883bcb79731216ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a04ae5f0bd5e4836883bcb79731216ef" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33445 } } }
I20260808 14:28:53.933352 10246 sys_catalog.cc:458] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef [sys.catalog]: This master's current role is: LEADER
W20260808 14:28:53.937803 10259 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:28:53.937919 10259 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:28:53.938027 10249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:28:53.938906 10249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:28:53.945050 10249 catalog_manager.cc:1347] Generated new cluster ID: db9be51330354c6aaad75100d1e7a406
I20260808 14:28:53.945130 10249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:28:53.966778 10249 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:28:53.967761 10249 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:28:53.978180 10249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a04ae5f0bd5e4836883bcb79731216ef: Generated new TSK 0
I20260808 14:28:53.979126 10249 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.125.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
I20260808 14:28:54.018654 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:33445
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:54.132210 10265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:54.132517 10265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:54.132566 10265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:54.137342 10265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:54.137403 10265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:54.137491 10265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:28:54.142762 10265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:33445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:54.143908 10265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:54.145013 10265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:28:54.154793 10265 server_base.cc:1034] running on GCE node
W20260808 14:28:54.154901 10271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:54.154886 10270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:54.154789 10273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:54.155483 10265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:54.156152 10265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:54.157341 10265 hybrid_clock.cc:648] HybridClock initialized: now 1786199334157323 us; error 31 us; skew 500 ppm
Aug 08 14:28:54 dist-test-slave-w206 krb5kdc[10155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199334, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:54.160282 10265 init.cc:377] Logged in from keytab as kudu/127.7.125.65@KRBTEST.COM (short username kudu)
I20260808 14:28:54.161325 10265 webserver.cc:460] Webserver started at http://127.7.125.65:41685/ using document root <none> and password file <none>
I20260808 14:28:54.161917 10265 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:54.162011 10265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:54.162253 10265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:54.164569 10265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "b80be234bad046efb686e85b5c1297a7"
format_stamp: "Formatted at 2026-08-08 14:28:54 on dist-test-slave-w206"
server_key: "9d88c16eb74949b630aceaac1d8fdae4"
server_key_iv: "4970256e5f7eb6469329e844060bee93"
server_key_version: "encryptionkey@0"
I20260808 14:28:54.165202 10265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "b80be234bad046efb686e85b5c1297a7"
format_stamp: "Formatted at 2026-08-08 14:28:54 on dist-test-slave-w206"
server_key: "9d88c16eb74949b630aceaac1d8fdae4"
server_key_iv: "4970256e5f7eb6469329e844060bee93"
server_key_version: "encryptionkey@0"
I20260808 14:28:54.169061 10265 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:28:54.171470 10281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:54.172689 10265 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:28:54.172847 10265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "b80be234bad046efb686e85b5c1297a7"
format_stamp: "Formatted at 2026-08-08 14:28:54 on dist-test-slave-w206"
server_key: "9d88c16eb74949b630aceaac1d8fdae4"
server_key_iv: "4970256e5f7eb6469329e844060bee93"
server_key_version: "encryptionkey@0"
I20260808 14:28:54.172966 10265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:54.188393 10265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:54.191510 10265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:54.191723 10265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:54.192337 10265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:28:54.193174 10265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:28:54.193248 10265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:54.193318 10265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:28:54.193363 10265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:54.202307 10265 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:38941
I20260808 14:28:54.202358 10400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:38941 every 8 connection(s)
I20260808 14:28:54.203250 10265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:28:54.206032 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 10265
I20260808 14:28:54.206146 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:28:54.206485 7669 external_mini_cluster.cc:1428] Setting key b7a2eb449d63639c1a86c08637a5f0ce
Aug 08 14:28:54 dist-test-slave-w206 krb5kdc[10155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199334, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.125.66@KRBTEST.COM; defaulting to no policy
I20260808 14:28:54.218603 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:54.204985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:42859 (local address 127.7.125.126:33445)
0808 14:28:54.205127 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:54.205131 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:54.206042 (+ 911us) server_negotiation.cc:407] Connection header received
0808 14:28:54.207597 (+ 1555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:54.207600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:54.207643 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:54.207714 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:54.209250 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:54.209771 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:54.211149 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:54.211579 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:54.211701 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:54.214493 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:54.214499 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:54.214505 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:54.214544 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:54.216435 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:54.216939 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:54.216943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:54.216945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:54.217048 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:54.217427 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:54.217433 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:54.217437 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:54.217695 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:54.217798 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:54.218347 (+ 549us) server_negotiation.cc:299] Negotiation successful
0808 14:28:54.218473 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 14:28:54.219525 10403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:54.205336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:33445 (local address 127.7.125.65:42859)
0808 14:28:54.205835 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:54.205903 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:54.207378 (+ 1475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:54.207920 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:54.207929 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:54.208409 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:54.209088 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:54.209112 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:54.209901 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:54.209907 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:54.211021 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:54.211039 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:54.211738 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:54.211741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:54.211888 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:54.212530 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:54.212547 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:54.214286 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:54.216558 (+ 2272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:54.216563 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:54.216576 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:54.216830 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:54.217152 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:54.217154 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:54.217155 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:54.217286 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:54.217799 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:54.217804 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:54.218072 (+ 268us) client_negotiation.cc:769] Sending connection context
0808 14:28:54.218258 (+ 186us) client_negotiation.cc:240] Negotiation successful
0808 14:28:54.218551 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":113,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.66@KRBTEST.COM" created.
I20260808 14:28:54.220881 10401 heartbeater.cc:347] Connected to a master server at 127.7.125.126:33445
I20260808 14:28:54.221146 10401 heartbeater.cc:464] Registering TS with master...
I20260808 14:28:54.221762 10401 heartbeater.cc:511] Master 127.7.125.126:33445 requested a full tablet report, sending...
I20260808 14:28:54.223625 10202 ts_manager.cc:194] Registered new tserver with Master: b80be234bad046efb686e85b5c1297a7 (127.7.125.65:38941)
I20260808 14:28:54.225301 10202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:42859
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
I20260808 14:28:54.239451 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:33445
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:54.354427 10406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:54.354691 10406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:54.354796 10406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:54.358892 10406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:54.358963 10406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:54.359143 10406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:28:54.364938 10406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:33445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:54.366036 10406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:54.367174 10406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:28:54.376786 10414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:54.376791 10411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:54.377060 10406 server_base.cc:1034] running on GCE node
W20260808 14:28:54.376822 10412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:54.377624 10406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:54.378294 10406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:54.379534 10406 hybrid_clock.cc:648] HybridClock initialized: now 1786199334379496 us; error 32 us; skew 500 ppm
Aug 08 14:28:54 dist-test-slave-w206 krb5kdc[10155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199334, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:54.382462 10406 init.cc:377] Logged in from keytab as kudu/127.7.125.66@KRBTEST.COM (short username kudu)
I20260808 14:28:54.383451 10406 webserver.cc:460] Webserver started at http://127.7.125.66:33101/ using document root <none> and password file <none>
I20260808 14:28:54.384038 10406 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:54.384131 10406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:54.384359 10406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:54.386488 10406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "eef0a869f4814333b8385b8501a9f386"
format_stamp: "Formatted at 2026-08-08 14:28:54 on dist-test-slave-w206"
server_key: "2d6731cde7f2e7de5b119cd8d505a2ae"
server_key_iv: "703b2b32ced56f505bf0b09d7ce747dd"
server_key_version: "encryptionkey@0"
I20260808 14:28:54.387135 10406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "eef0a869f4814333b8385b8501a9f386"
format_stamp: "Formatted at 2026-08-08 14:28:54 on dist-test-slave-w206"
server_key: "2d6731cde7f2e7de5b119cd8d505a2ae"
server_key_iv: "703b2b32ced56f505bf0b09d7ce747dd"
server_key_version: "encryptionkey@0"
I20260808 14:28:54.390823 10406 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:28:54.393345 10422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:54.394528 10406 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:28:54.394666 10406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "eef0a869f4814333b8385b8501a9f386"
format_stamp: "Formatted at 2026-08-08 14:28:54 on dist-test-slave-w206"
server_key: "2d6731cde7f2e7de5b119cd8d505a2ae"
server_key_iv: "703b2b32ced56f505bf0b09d7ce747dd"
server_key_version: "encryptionkey@0"
I20260808 14:28:54.394779 10406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:54.409641 10406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:54.412842 10406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:54.413050 10406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:54.413746 10406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:28:54.414757 10406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:28:54.414840 10406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:54.414925 10406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:28:54.414960 10406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:54.426153 10406 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:34837
I20260808 14:28:54.426179 10541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:34837 every 8 connection(s)
I20260808 14:28:54.427243 10406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
Aug 08 14:28:54 dist-test-slave-w206 krb5kdc[10155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199334, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:54.436964 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 10406
I20260808 14:28:54.437078 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:28:54.437412 7669 external_mini_cluster.cc:1428] Setting key 074d1be7cdd8cdf4713bb6f2ff2f8884
I20260808 14:28:54.441509 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:54.429077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:59041 (local address 127.7.125.126:33445)
0808 14:28:54.429228 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:54.429232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:54.430118 (+ 886us) server_negotiation.cc:407] Connection header received
0808 14:28:54.431411 (+ 1293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:54.431414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:54.431462 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:54.431540 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:54.432777 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:54.433353 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:54.434599 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:54.434837 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:54.434954 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:54.437529 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:54.437532 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:54.437534 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:54.437557 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:54.439344 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:54.439846 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:54.439853 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:54.439858 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:54.439974 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:54.440308 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:54.440313 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:54.440318 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:54.440578 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:54.440689 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:54.441248 (+ 559us) server_negotiation.cc:299] Negotiation successful
0808 14:28:54.441381 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:28:54.442674 10544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:54.429514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:33445 (local address 127.7.125.66:59041)
0808 14:28:54.429969 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:54.430011 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:54.431203 (+ 1192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:54.431695 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:54.431702 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:54.432079 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:54.432645 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:54.432661 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:54.433574 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:54.433577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:54.434475 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:54.434485 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:54.435023 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:54.435029 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:54.435198 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:54.435732 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:54.435747 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:54.437333 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:54.439470 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:54.439475 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:54.439488 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:54.439741 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:54.440076 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:54.440079 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:54.440080 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:54.440214 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:54.440685 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:54.440690 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:54.440962 (+ 272us) client_negotiation.cc:769] Sending connection context
0808 14:28:54.441172 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 14:28:54.441556 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":117,"threads_started":1}
I20260808 14:28:54.443928 10542 heartbeater.cc:347] Connected to a master server at 127.7.125.126:33445
I20260808 14:28:54.444195 10542 heartbeater.cc:464] Registering TS with master...
I20260808 14:28:54.444788 10542 heartbeater.cc:511] Master 127.7.125.126:33445 requested a full tablet report, sending...
I20260808 14:28:54.445919 10202 ts_manager.cc:194] Registered new tserver with Master: eef0a869f4814333b8385b8501a9f386 (127.7.125.66:34837)
I20260808 14:28:54.446497 10202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:59041
WARNING: no policy specified for kudu/127.7.125.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
I20260808 14:28:54.466667 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:33445
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:54.577849 10547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:54.578105 10547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:54.578147 10547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:54.582201 10547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:54.582257 10547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:54.582321 10547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:28:54.587626 10547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:33445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:54.588586 10547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:54.589516 10547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:28:54.598730 10555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:54.598724 10553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:54.598692 10552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:54.599038 10547 server_base.cc:1034] running on GCE node
I20260808 14:28:54.599423 10547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:54.600085 10547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:54.601245 10547 hybrid_clock.cc:648] HybridClock initialized: now 1786199334601215 us; error 42 us; skew 500 ppm
Aug 08 14:28:54 dist-test-slave-w206 krb5kdc[10155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199334, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:54.604153 10547 init.cc:377] Logged in from keytab as kudu/127.7.125.67@KRBTEST.COM (short username kudu)
I20260808 14:28:54.605039 10547 webserver.cc:460] Webserver started at http://127.7.125.67:38827/ using document root <none> and password file <none>
I20260808 14:28:54.605557 10547 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:54.605615 10547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:54.605777 10547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:54.607824 10547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "9f3c54c3c79b4df3abbce7f7783b984d"
format_stamp: "Formatted at 2026-08-08 14:28:54 on dist-test-slave-w206"
server_key: "4c0060735fb087a0911fe334d8f52210"
server_key_iv: "abf6855843c508f316e91847403c252c"
server_key_version: "encryptionkey@0"
I20260808 14:28:54.608348 10547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "9f3c54c3c79b4df3abbce7f7783b984d"
format_stamp: "Formatted at 2026-08-08 14:28:54 on dist-test-slave-w206"
server_key: "4c0060735fb087a0911fe334d8f52210"
server_key_iv: "abf6855843c508f316e91847403c252c"
server_key_version: "encryptionkey@0"
I20260808 14:28:54.611938 10547 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:28:54.614255 10562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:54.615540 10547 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:28:54.615664 10547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "9f3c54c3c79b4df3abbce7f7783b984d"
format_stamp: "Formatted at 2026-08-08 14:28:54 on dist-test-slave-w206"
server_key: "4c0060735fb087a0911fe334d8f52210"
server_key_iv: "abf6855843c508f316e91847403c252c"
server_key_version: "encryptionkey@0"
I20260808 14:28:54.615770 10547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:54.630781 10547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:54.634341 10547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:54.634536 10547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:54.635190 10547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:28:54.636011 10547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:28:54.636058 10547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:54.636123 10547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:28:54.636158 10547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:54.645794 10547 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:36601
I20260808 14:28:54.645808 10682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:36601 every 8 connection(s)
I20260808 14:28:54.646715 10547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:28:54.653051 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 10547
I20260808 14:28:54.653188 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:28:54.653488 7669 external_mini_cluster.cc:1428] Setting key 662a4a59759aad8abb35c91ef2df083a
Aug 08 14:28:54 dist-test-slave-w206 krb5kdc[10155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199334, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:54.661645 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:54.648550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:43137 (local address 127.7.125.126:33445)
0808 14:28:54.648698 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:54.648702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:54.649727 (+ 1025us) server_negotiation.cc:407] Connection header received
0808 14:28:54.651230 (+ 1503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:54.651234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:54.651297 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:54.651417 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:54.652653 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:54.653341 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:54.654596 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:54.654842 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:54.654968 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:54.657675 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:54.657684 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:54.657686 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:54.657711 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:54.659394 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:54.659970 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:54.659976 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:54.659978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:54.660086 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:54.660466 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:54.660471 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:54.660473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:54.660673 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:54.660789 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:54.661321 (+ 532us) server_negotiation.cc:299] Negotiation successful
0808 14:28:54.661455 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 14:28:54.662434 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:54.648937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:33445 (local address 127.7.125.67:43137)
0808 14:28:54.649466 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:54.649512 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:54.650913 (+ 1401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:54.651598 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:54.651605 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:54.651980 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:54.652454 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:54.652467 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:54.653484 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:54.653488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:54.654456 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:54.654466 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:54.655032 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:54.655038 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:54.655234 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:54.655898 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:54.655916 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:54.657456 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:54.659525 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:54.659537 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:54.659552 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:54.659845 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:54.660198 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:54.660202 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:54.660204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:54.660360 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:54.660798 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:54.660807 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:54.661063 (+ 256us) client_negotiation.cc:769] Sending connection context
0808 14:28:54.661259 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 14:28:54.661595 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":107,"threads_started":1}
I20260808 14:28:54.663654 10683 heartbeater.cc:347] Connected to a master server at 127.7.125.126:33445
I20260808 14:28:54.663908 10683 heartbeater.cc:464] Registering TS with master...
I20260808 14:28:54.664464 10683 heartbeater.cc:511] Master 127.7.125.126:33445 requested a full tablet report, sending...
I20260808 14:28:54.665493 10202 ts_manager.cc:194] Registered new tserver with Master: 9f3c54c3c79b4df3abbce7f7783b984d (127.7.125.67:36601)
I20260808 14:28:54.666208 10202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:43137
I20260808 14:28:54.667452 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 14:28:54.685922 10241 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:28:54.684353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44390 (local address 127.7.125.126:33445)
0808 14:28:54.684531 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:54.684537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:54.684684 (+ 147us) server_negotiation.cc:407] Connection header received
0808 14:28:54.684879 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:54.684882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:54.684924 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:54.685012 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:54.685756 (+ 744us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44390: BlockingRecv error: recv got EOF from 127.0.0.1:44390 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 14:28:54.686936 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 10265
I20260808 14:28:54.693945 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 10406
I20260808 14:28:54.700735 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 10547
I20260808 14:28:54.706555 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 10170
2026-08-08T14:28:54Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2204 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:28:54 dist-test-slave-w206 krb5kdc[10695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:28:54 dist-test-slave-w206 krb5kdc[10695](info): set up 2 sockets
Aug 08 14:28:54 dist-test-slave-w206 krb5kdc[10695](info): commencing operation
krb5kdc: starting...
W20260808 14:28:55.787627 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:28:55 dist-test-slave-w206 krb5kdc[10695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:28:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:28:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:28:55.918620 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:39415
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:36957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:39415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:56.032603 10709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:56.032907 10709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:56.032995 10709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:56.037002 10709 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:28:56.037091 10709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:56.037130 10709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:56.037168 10709 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:28:56.037200 10709 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:28:56.042838 10709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:36957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:39415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:39415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:56.044020 10709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:56.045074 10709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:28:56.053691 10714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:56.053719 10709 server_base.cc:1034] running on GCE node
W20260808 14:28:56.053647 10715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:56.053647 10717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:56.054662 10709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:56.055863 10709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:56.057032 10709 hybrid_clock.cc:648] HybridClock initialized: now 1786199336057017 us; error 35 us; skew 500 ppm
Aug 08 14:28:56 dist-test-slave-w206 krb5kdc[10695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199336, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:56.060005 10709 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:28:56.060916 10709 webserver.cc:460] Webserver started at http://127.7.125.126:33271/ using document root <none> and password file <none>
I20260808 14:28:56.061479 10709 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:56.061574 10709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:56.061813 10709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:56.063982 10709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "38a005b3db9e48288716478a4eff0c3b"
format_stamp: "Formatted at 2026-08-08 14:28:56 on dist-test-slave-w206"
server_key: "bd2d4611a5b41b1315e4e2ec1130d272"
server_key_iv: "71ca76d651214b23c1ec91a0e4c85809"
server_key_version: "encryptionkey@0"
I20260808 14:28:56.064566 10709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "38a005b3db9e48288716478a4eff0c3b"
format_stamp: "Formatted at 2026-08-08 14:28:56 on dist-test-slave-w206"
server_key: "bd2d4611a5b41b1315e4e2ec1130d272"
server_key_iv: "71ca76d651214b23c1ec91a0e4c85809"
server_key_version: "encryptionkey@0"
I20260808 14:28:56.068388 10709 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:28:56.070837 10725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:56.072170 10709 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:28:56.072331 10709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "38a005b3db9e48288716478a4eff0c3b"
format_stamp: "Formatted at 2026-08-08 14:28:56 on dist-test-slave-w206"
server_key: "bd2d4611a5b41b1315e4e2ec1130d272"
server_key_iv: "71ca76d651214b23c1ec91a0e4c85809"
server_key_version: "encryptionkey@0"
I20260808 14:28:56.072463 10709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:56.085114 10709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:56.088558 10709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:56.088773 10709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:56.095929 10709 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:39415
I20260808 14:28:56.096243 10778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:39415 every 8 connection(s)
I20260808 14:28:56.096948 10709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:28:56.099787 10779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:56.105177 10779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b: Bootstrap starting.
I20260808 14:28:56.105553 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 10709
I20260808 14:28:56.105683 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:28:56.105971 7669 external_mini_cluster.cc:1428] Setting key 97076c3b8f9e31393fcec8c63b1af858
I20260808 14:28:56.108058 10779 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:56.108814 10779 log.cc:824] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b: Log is configured to *not* fsync() on all Append() calls
I20260808 14:28:56.110673 10779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b: No bootstrap required, opened a new log
I20260808 14:28:56.114179 10779 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a005b3db9e48288716478a4eff0c3b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39415 } }
I20260808 14:28:56.114502 10779 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:56.114563 10779 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a005b3db9e48288716478a4eff0c3b, State: Initialized, Role: FOLLOWER
Aug 08 14:28:56 dist-test-slave-w206 krb5kdc[10695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:56.115118 10779 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a005b3db9e48288716478a4eff0c3b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39415 } }
I20260808 14:28:56.115300 10779 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:28:56.115377 10779 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:28:56.115499 10779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:56.116475 10779 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a005b3db9e48288716478a4eff0c3b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39415 } }
I20260808 14:28:56.116920 10779 leader_election.cc:302] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38a005b3db9e48288716478a4eff0c3b; no voters:
I20260808 14:28:56.117321 10779 leader_election.cc:288] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:28:56.117568 10784 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:28:56.117838 10784 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b [term 1 LEADER]: Becoming Leader. State: Replica: 38a005b3db9e48288716478a4eff0c3b, State: Running, Role: LEADER
I20260808 14:28:56.118170 10784 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a005b3db9e48288716478a4eff0c3b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39415 } }
I20260808 14:28:56.118803 10779 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:28:56.119462 10786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38a005b3db9e48288716478a4eff0c3b. Latest consensus state: current_term: 1 leader_uuid: "38a005b3db9e48288716478a4eff0c3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a005b3db9e48288716478a4eff0c3b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39415 } } }
I20260808 14:28:56.119601 10786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b [sys.catalog]: This master's current role is: LEADER
I20260808 14:28:56.119966 10785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38a005b3db9e48288716478a4eff0c3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a005b3db9e48288716478a4eff0c3b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39415 } } }
I20260808 14:28:56.120076 10785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b [sys.catalog]: This master's current role is: LEADER
I20260808 14:28:56.121693 10782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:56.107306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60278 (local address 127.7.125.126:39415)
0808 14:28:56.107751 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:56.107766 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:56.107803 (+ 37us) server_negotiation.cc:407] Connection header received
0808 14:28:56.108518 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:56.108537 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:56.108863 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:56.109235 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:56.110119 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:56.111198 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:56.112129 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:56.112612 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:56.112781 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:56.115649 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:56.115656 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:56.115679 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:56.115707 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:56.117856 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:56.118420 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:56.118427 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:56.118434 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:56.118576 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:56.119008 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:56.119012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:56.119014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:56.119564 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:56.119766 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:56.120307 (+ 541us) server_negotiation.cc:299] Negotiation successful
0808 14:28:56.120628 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":134,"threads_started":1}
W20260808 14:28:56.124316 10801 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:28:56.124397 10801 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:28:56.124466 10790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:28:56.125305 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:28:56.125340 10790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:28:56.131357 10790 catalog_manager.cc:1347] Generated new cluster ID: 88f85c5f53bd4e32a1b0e7bf0db1a116
I20260808 14:28:56.131424 10790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:28:56.146351 10782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:56.140746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60282 (local address 127.7.125.126:39415)
0808 14:28:56.140891 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:56.140896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:56.141092 (+ 196us) server_negotiation.cc:407] Connection header received
0808 14:28:56.141194 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:56.141197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:56.141248 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:56.141352 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:56.141900 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:56.142713 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:56.143865 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:56.144231 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:56.144341 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:56.145270 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:56.145274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:56.145277 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:28:56.145891 (+ 614us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:56.145928 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:28:56.145949 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:56.146050 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:56.146135 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 14:28:56.146200 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 14:28:56.161554 10790 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:28:56.162547 10790 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:28:56.171209 10790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38a005b3db9e48288716478a4eff0c3b: Generated new TSK 0
I20260808 14:28:56.171962 10790 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:28:56.180296 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 10709
2026-08-08T14:28:56Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1471 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:28:56 dist-test-slave-w206 krb5kdc[10814](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:28:56 dist-test-slave-w206 krb5kdc[10814](info): set up 2 sockets
Aug 08 14:28:56 dist-test-slave-w206 krb5kdc[10814](info): commencing operation
krb5kdc: starting...
W20260808 14:28:57.255240 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:28:57 dist-test-slave-w206 krb5kdc[10814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:28:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:28:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:28:57.382226 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:42917
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:46763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:42917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:57.497530 10829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:57.497872 10829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:57.497957 10829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:57.502085 10829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:28:57.502207 10829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:57.502252 10829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:57.502318 10829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:28:57.502360 10829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:28:57.507737 10829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:46763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:42917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:42917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:57.508853 10829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:57.510087 10829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:28:57.519042 10834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:57.519061 10838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:57.519054 10836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:57.519124 10829 server_base.cc:1034] running on GCE node
I20260808 14:28:57.519819 10829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:57.520965 10829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:57.522166 10829 hybrid_clock.cc:648] HybridClock initialized: now 1786199337522132 us; error 45 us; skew 500 ppm
Aug 08 14:28:57 dist-test-slave-w206 krb5kdc[10814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199337, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:57.525184 10829 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:28:57.526090 10829 webserver.cc:460] Webserver started at http://127.7.125.126:38131/ using document root <none> and password file <none>
I20260808 14:28:57.526643 10829 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:57.526703 10829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:57.526947 10829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:57.529309 10829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "dd4e47ee55f9400da8a00d57774f919b"
format_stamp: "Formatted at 2026-08-08 14:28:57 on dist-test-slave-w206"
server_key: "e97617c36fd3f1d3ef61c1fdd202c70b"
server_key_iv: "9ac81b39b9d534663f219a6aa9e646c4"
server_key_version: "encryptionkey@0"
I20260808 14:28:57.529953 10829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "dd4e47ee55f9400da8a00d57774f919b"
format_stamp: "Formatted at 2026-08-08 14:28:57 on dist-test-slave-w206"
server_key: "e97617c36fd3f1d3ef61c1fdd202c70b"
server_key_iv: "9ac81b39b9d534663f219a6aa9e646c4"
server_key_version: "encryptionkey@0"
I20260808 14:28:57.533897 10829 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260808 14:28:57.536378 10845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:57.537599 10829 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:28:57.537740 10829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "dd4e47ee55f9400da8a00d57774f919b"
format_stamp: "Formatted at 2026-08-08 14:28:57 on dist-test-slave-w206"
server_key: "e97617c36fd3f1d3ef61c1fdd202c70b"
server_key_iv: "9ac81b39b9d534663f219a6aa9e646c4"
server_key_version: "encryptionkey@0"
I20260808 14:28:57.537855 10829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:57.572742 10829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:57.575843 10829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:57.576058 10829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:57.583586 10829 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:42917
I20260808 14:28:57.583865 10899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:42917 every 8 connection(s)
I20260808 14:28:57.584723 10829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:28:57.587598 10901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:57.589790 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 10829
I20260808 14:28:57.589949 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:28:57.590224 7669 external_mini_cluster.cc:1428] Setting key c35c3de945f9dbf9c54bebd7f828ed21
I20260808 14:28:57.593683 10901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b: Bootstrap starting.
I20260808 14:28:57.595947 10901 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:57.596680 10901 log.cc:824] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b: Log is configured to *not* fsync() on all Append() calls
I20260808 14:28:57.598572 10901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b: No bootstrap required, opened a new log
Aug 08 14:28:57 dist-test-slave-w206 krb5kdc[10814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:57.601773 10901 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd4e47ee55f9400da8a00d57774f919b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 42917 } }
I20260808 14:28:57.602005 10901 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:57.602065 10901 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd4e47ee55f9400da8a00d57774f919b, State: Initialized, Role: FOLLOWER
I20260808 14:28:57.602491 10901 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd4e47ee55f9400da8a00d57774f919b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 42917 } }
I20260808 14:28:57.602608 10901 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:28:57.602708 10901 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:28:57.602821 10901 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:57.603717 10901 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd4e47ee55f9400da8a00d57774f919b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 42917 } }
I20260808 14:28:57.604054 10901 leader_election.cc:302] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd4e47ee55f9400da8a00d57774f919b; no voters:
I20260808 14:28:57.604377 10901 leader_election.cc:288] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:28:57.604458 10906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:28:57.604655 10906 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b [term 1 LEADER]: Becoming Leader. State: Replica: dd4e47ee55f9400da8a00d57774f919b, State: Running, Role: LEADER
I20260808 14:28:57.605000 10906 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd4e47ee55f9400da8a00d57774f919b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 42917 } }
I20260808 14:28:57.605655 10901 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:28:57.606230 10904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:57.591710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58766 (local address 127.7.125.126:42917)
0808 14:28:57.592160 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:57.592176 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:57.592212 (+ 36us) server_negotiation.cc:407] Connection header received
0808 14:28:57.592874 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:57.592892 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:57.593224 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:57.593567 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:57.594397 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:57.595487 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:57.596425 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:57.596939 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:57.597106 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:57.599872 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:57.599877 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:57.599890 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:57.599927 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:57.602728 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:57.603282 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:57.603289 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:57.603295 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:57.603425 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:57.603725 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:57.603729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:57.603730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:57.604213 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:57.604435 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:57.605070 (+ 635us) server_negotiation.cc:299] Negotiation successful
0808 14:28:57.605349 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":129,"threads_started":1}
I20260808 14:28:57.606873 10907 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd4e47ee55f9400da8a00d57774f919b. Latest consensus state: current_term: 1 leader_uuid: "dd4e47ee55f9400da8a00d57774f919b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd4e47ee55f9400da8a00d57774f919b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 42917 } } }
I20260808 14:28:57.607033 10907 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b [sys.catalog]: This master's current role is: LEADER
I20260808 14:28:57.607362 10908 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd4e47ee55f9400da8a00d57774f919b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd4e47ee55f9400da8a00d57774f919b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 42917 } } }
I20260808 14:28:57.607482 10908 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b [sys.catalog]: This master's current role is: LEADER
W20260808 14:28:57.610935 10921 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:28:57.611038 10921 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:28:57.611122 10915 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:28:57.611775 10915 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:28:57.616741 10915 catalog_manager.cc:1347] Generated new cluster ID: 76a68ad3740a4014ab1e74b1cca3abd5
I20260808 14:28:57.616806 10915 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:28:57.635878 10915 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:28:57.637172 10915 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:28:57.645614 10915 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd4e47ee55f9400da8a00d57774f919b: Generated new TSK 0
I20260808 14:28:57.646432 10915 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.125.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
I20260808 14:28:57.677685 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:42917
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:46763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:57.789244 10927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:57.789482 10927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:57.789523 10927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:57.793474 10927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:57.793531 10927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:57.793587 10927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:28:57.798730 10927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:46763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:42917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:57.799937 10927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:57.800933 10927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:28:57.811367 10932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:57.811388 10935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:57.811568 10933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:57.811995 10927 server_base.cc:1034] running on GCE node
I20260808 14:28:57.812475 10927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:57.813164 10927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:57.814363 10927 hybrid_clock.cc:648] HybridClock initialized: now 1786199337814340 us; error 33 us; skew 500 ppm
Aug 08 14:28:57 dist-test-slave-w206 krb5kdc[10814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199337, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:57.817755 10927 init.cc:377] Logged in from keytab as kudu/127.7.125.65@KRBTEST.COM (short username kudu)
I20260808 14:28:57.818735 10927 webserver.cc:460] Webserver started at http://127.7.125.65:33545/ using document root <none> and password file <none>
I20260808 14:28:57.819404 10927 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:57.819470 10927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:57.819729 10927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:57.821853 10927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "b76d1b998f8d4a7dbb10e55b0dee1eb2"
format_stamp: "Formatted at 2026-08-08 14:28:57 on dist-test-slave-w206"
server_key: "c6cd128c555421de2778911c91ba288a"
server_key_iv: "795e6ea4248a5148186365cfd4bc6aef"
server_key_version: "encryptionkey@0"
I20260808 14:28:57.822513 10927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "b76d1b998f8d4a7dbb10e55b0dee1eb2"
format_stamp: "Formatted at 2026-08-08 14:28:57 on dist-test-slave-w206"
server_key: "c6cd128c555421de2778911c91ba288a"
server_key_iv: "795e6ea4248a5148186365cfd4bc6aef"
server_key_version: "encryptionkey@0"
I20260808 14:28:57.826519 10927 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:28:57.829113 10943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:57.830415 10927 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:28:57.830550 10927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "b76d1b998f8d4a7dbb10e55b0dee1eb2"
format_stamp: "Formatted at 2026-08-08 14:28:57 on dist-test-slave-w206"
server_key: "c6cd128c555421de2778911c91ba288a"
server_key_iv: "795e6ea4248a5148186365cfd4bc6aef"
server_key_version: "encryptionkey@0"
I20260808 14:28:57.830660 10927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:57.873144 10927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:57.877099 10927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:57.877347 10927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:57.878037 10927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:28:57.878945 10927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:28:57.879017 10927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:57.879097 10927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:28:57.879132 10927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:57.889004 10927 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:33489
I20260808 14:28:57.889029 11061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:33489 every 8 connection(s)
I20260808 14:28:57.890008 10927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:28:57.895241 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 10927
I20260808 14:28:57.895391 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:28:57.895676 7669 external_mini_cluster.cc:1428] Setting key ece738a67f7e0bf40d52bb36bb9002a0
Aug 08 14:28:57 dist-test-slave-w206 krb5kdc[10814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199337, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:57.905545 10904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:57.891956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:48519 (local address 127.7.125.126:42917)
0808 14:28:57.892109 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:57.892112 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:57.893028 (+ 916us) server_negotiation.cc:407] Connection header received
0808 14:28:57.894408 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:57.894412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:57.894484 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:57.894580 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:57.895979 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:57.896608 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:57.898010 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:57.898441 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:57.898570 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:57.901296 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:57.901303 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:57.901309 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:57.901342 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:57.903205 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:57.903742 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:57.903747 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:57.903752 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:57.903855 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:57.904199 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:57.904204 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:57.904208 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:57.904493 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:57.904606 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:57.905133 (+ 527us) server_negotiation.cc:299] Negotiation successful
0808 14:28:57.905321 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 14:28:57.906394 11064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:57.892384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:42917 (local address 127.7.125.65:48519)
0808 14:28:57.892851 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:57.892902 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:57.894169 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:57.894760 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:57.894767 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:57.895207 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:57.895821 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:57.895838 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:57.896784 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:57.896787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:57.897866 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:57.897877 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:57.898607 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:57.898610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:57.898782 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:57.899574 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:57.899591 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:57.901112 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:57.903326 (+ 2214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:57.903331 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:57.903367 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:57.903631 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:57.903959 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:57.903962 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:57.903963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:57.904096 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:57.904614 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:57.904619 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:57.904885 (+ 266us) client_negotiation.cc:769] Sending connection context
0808 14:28:57.905047 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 14:28:57.905371 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":114,"threads_started":1}
WARNING: no policy specified for kudu/127.7.125.66@KRBTEST.COM; defaulting to no policy
I20260808 14:28:57.907792 11062 heartbeater.cc:347] Connected to a master server at 127.7.125.126:42917
I20260808 14:28:57.908037 11062 heartbeater.cc:464] Registering TS with master...
I20260808 14:28:57.908640 11062 heartbeater.cc:511] Master 127.7.125.126:42917 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.66@KRBTEST.COM" created.
I20260808 14:28:57.910197 10863 ts_manager.cc:194] Registered new tserver with Master: b76d1b998f8d4a7dbb10e55b0dee1eb2 (127.7.125.65:33489)
I20260808 14:28:57.911454 10863 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:48519
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
I20260808 14:28:57.927510 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:42917
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:46763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:58.044232 11067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:58.044541 11067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:58.044627 11067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:58.048689 11067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:58.048794 11067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:58.048898 11067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:28:58.054245 11067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:46763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:42917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:58.055481 11067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:58.056567 11067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:28:58.066886 11073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:58.066886 11072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:58.067023 11075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:58.067010 11067 server_base.cc:1034] running on GCE node
I20260808 14:28:58.067631 11067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:58.068298 11067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:58.069604 11067 hybrid_clock.cc:648] HybridClock initialized: now 1786199338069571 us; error 30 us; skew 500 ppm
Aug 08 14:28:58 dist-test-slave-w206 krb5kdc[10814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199338, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:58.072956 11067 init.cc:377] Logged in from keytab as kudu/127.7.125.66@KRBTEST.COM (short username kudu)
I20260808 14:28:58.073969 11067 webserver.cc:460] Webserver started at http://127.7.125.66:34119/ using document root <none> and password file <none>
I20260808 14:28:58.074517 11067 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:58.074580 11067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:58.074761 11067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:58.077318 11067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "e18f2283b3174660a824a983e4863c70"
format_stamp: "Formatted at 2026-08-08 14:28:58 on dist-test-slave-w206"
server_key: "020fc20ede8e910854b9f588430fbf42"
server_key_iv: "900894b4f941dd7273b358055c0b882b"
server_key_version: "encryptionkey@0"
I20260808 14:28:58.077970 11067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "e18f2283b3174660a824a983e4863c70"
format_stamp: "Formatted at 2026-08-08 14:28:58 on dist-test-slave-w206"
server_key: "020fc20ede8e910854b9f588430fbf42"
server_key_iv: "900894b4f941dd7273b358055c0b882b"
server_key_version: "encryptionkey@0"
I20260808 14:28:58.082106 11067 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260808 14:28:58.084924 11082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:58.086208 11067 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:28:58.086321 11067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "e18f2283b3174660a824a983e4863c70"
format_stamp: "Formatted at 2026-08-08 14:28:58 on dist-test-slave-w206"
server_key: "020fc20ede8e910854b9f588430fbf42"
server_key_iv: "900894b4f941dd7273b358055c0b882b"
server_key_version: "encryptionkey@0"
I20260808 14:28:58.086431 11067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:58.107699 11067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:58.111275 11067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:58.111496 11067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:58.112118 11067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:28:58.112988 11067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:28:58.113039 11067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:58.113108 11067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:28:58.113152 11067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:58.122105 11067 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:40757
I20260808 14:28:58.122189 11203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:40757 every 8 connection(s)
I20260808 14:28:58.123155 11067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:28:58.124351 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 11067
I20260808 14:28:58.124469 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:28:58.124797 7669 external_mini_cluster.cc:1428] Setting key 2825e824f4a4bb227e93dfa269259568
Aug 08 14:28:58 dist-test-slave-w206 krb5kdc[10814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199338, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.125.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.67@KRBTEST.COM" created.
I20260808 14:28:58.139490 10904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:58.125335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:59965 (local address 127.7.125.126:42917)
0808 14:28:58.125485 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:58.125490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:58.126368 (+ 878us) server_negotiation.cc:407] Connection header received
0808 14:28:58.127910 (+ 1542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:58.127913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:58.127969 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:58.128051 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:58.129720 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:58.130267 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:58.132196 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:58.132419 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:58.132504 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:58.135780 (+ 3276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:58.135783 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:58.135785 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:58.135824 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:58.137424 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:58.137902 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:58.137905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:58.137906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:58.137978 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:58.138275 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:58.138277 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:58.138279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:58.138502 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:58.138615 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:58.139209 (+ 594us) server_negotiation.cc:299] Negotiation successful
0808 14:28:58.139340 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 14:28:58.140411 11206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:58.125668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:42917 (local address 127.7.125.66:59965)
0808 14:28:58.126217 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:58.126278 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:58.127645 (+ 1367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:58.128227 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:58.128234 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:58.128790 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:58.129517 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:58.129543 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:58.130417 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:58.130423 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:58.132066 (+ 1643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:58.132078 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:58.132600 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:58.132607 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:58.132854 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:58.133585 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:58.133613 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:58.135587 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:58.137532 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:58.137537 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:58.137551 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:58.137804 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:58.138055 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:58.138057 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:58.138059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:58.138192 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:58.138626 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:58.138632 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:58.138870 (+ 238us) client_negotiation.cc:769] Sending connection context
0808 14:28:58.139119 (+ 249us) client_negotiation.cc:240] Negotiation successful
0808 14:28:58.139501 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":119,"threads_started":1}
I20260808 14:28:58.142216 11204 heartbeater.cc:347] Connected to a master server at 127.7.125.126:42917
I20260808 14:28:58.142520 11204 heartbeater.cc:464] Registering TS with master...
I20260808 14:28:58.143124 11204 heartbeater.cc:511] Master 127.7.125.126:42917 requested a full tablet report, sending...
I20260808 14:28:58.144246 10863 ts_manager.cc:194] Registered new tserver with Master: e18f2283b3174660a824a983e4863c70 (127.7.125.66:40757)
I20260808 14:28:58.144829 10863 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:59965
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
I20260808 14:28:58.157955 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:42917
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:46763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:58.274573 11209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:58.274828 11209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:58.274871 11209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:58.278842 11209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:58.278898 11209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:58.279021 11209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:28:58.284250 11209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:46763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:42917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:58.285316 11209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:58.286319 11209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:28:58.296655 11215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:58.296861 11209 server_base.cc:1034] running on GCE node
W20260808 14:28:58.296682 11214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:58.296666 11217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:58.297353 11209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:58.298043 11209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:58.299202 11209 hybrid_clock.cc:648] HybridClock initialized: now 1786199338299161 us; error 52 us; skew 500 ppm
Aug 08 14:28:58 dist-test-slave-w206 krb5kdc[10814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199338, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:58.302186 11209 init.cc:377] Logged in from keytab as kudu/127.7.125.67@KRBTEST.COM (short username kudu)
I20260808 14:28:58.303104 11209 webserver.cc:460] Webserver started at http://127.7.125.67:34087/ using document root <none> and password file <none>
I20260808 14:28:58.303642 11209 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:58.303699 11209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:58.303875 11209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:58.305927 11209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "72247991c7ae4381a4d7eee09020a200"
format_stamp: "Formatted at 2026-08-08 14:28:58 on dist-test-slave-w206"
server_key: "c034164c0d5c932a6e7954aa6b084e12"
server_key_iv: "a93f49d3e81ebd32c59cf1632be49e2f"
server_key_version: "encryptionkey@0"
I20260808 14:28:58.306552 11209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "72247991c7ae4381a4d7eee09020a200"
format_stamp: "Formatted at 2026-08-08 14:28:58 on dist-test-slave-w206"
server_key: "c034164c0d5c932a6e7954aa6b084e12"
server_key_iv: "a93f49d3e81ebd32c59cf1632be49e2f"
server_key_version: "encryptionkey@0"
I20260808 14:28:58.310458 11209 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260808 14:28:58.312995 11224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:58.314407 11209 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:28:58.314545 11209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "72247991c7ae4381a4d7eee09020a200"
format_stamp: "Formatted at 2026-08-08 14:28:58 on dist-test-slave-w206"
server_key: "c034164c0d5c932a6e7954aa6b084e12"
server_key_iv: "a93f49d3e81ebd32c59cf1632be49e2f"
server_key_version: "encryptionkey@0"
I20260808 14:28:58.314659 11209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:58.349663 11209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:58.353542 11209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:58.353765 11209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:58.354399 11209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:28:58.355329 11209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:28:58.355403 11209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:58.355479 11209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:28:58.355522 11209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:58.364745 11209 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:37989
I20260808 14:28:58.364778 11345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:37989 every 8 connection(s)
I20260808 14:28:58.365649 11209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
Aug 08 14:28:58 dist-test-slave-w206 krb5kdc[10814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199338, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:58.375936 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 11209
I20260808 14:28:58.376086 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:28:58.376384 7669 external_mini_cluster.cc:1428] Setting key ea1e3c662776b90044537e8041226438
I20260808 14:28:58.380097 10904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:58.367577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:53085 (local address 127.7.125.126:42917)
0808 14:28:58.367723 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:58.367727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:58.368481 (+ 754us) server_negotiation.cc:407] Connection header received
0808 14:28:58.369896 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:58.369899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:58.369959 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:58.370069 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:58.371321 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:58.371875 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:58.373019 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:58.373291 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:58.373412 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:58.376067 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:58.376072 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:58.376074 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:58.376113 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:58.377908 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:58.378407 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:58.378411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:58.378412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:58.378514 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:58.378927 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:58.378930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:58.378931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:58.379219 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:58.379361 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:58.379844 (+ 483us) server_negotiation.cc:299] Negotiation successful
0808 14:28:58.379976 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:28:58.381033 11348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:58.367822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:42917 (local address 127.7.125.67:53085)
0808 14:28:58.368317 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 14:28:58.368363 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 14:28:58.369667 (+ 1304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:28:58.370247 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:28:58.370254 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:28:58.370616 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:28:58.371174 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:58.371191 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:58.372022 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:58.372025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:58.372885 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:28:58.372899 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:28:58.373456 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:28:58.373460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:28:58.373629 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:58.374273 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:28:58.374289 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:28:58.375866 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:28:58.378022 (+ 2156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:58.378027 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:58.378040 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:58.378305 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:58.378610 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:28:58.378612 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:28:58.378614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:28:58.378807 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:28:58.379366 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:28:58.379371 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:28:58.379618 (+ 247us) client_negotiation.cc:769] Sending connection context
0808 14:28:58.379783 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 14:28:58.380081 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":121,"threads_started":1}
I20260808 14:28:58.382166 11346 heartbeater.cc:347] Connected to a master server at 127.7.125.126:42917
I20260808 14:28:58.382407 11346 heartbeater.cc:464] Registering TS with master...
I20260808 14:28:58.382910 11346 heartbeater.cc:511] Master 127.7.125.126:42917 requested a full tablet report, sending...
I20260808 14:28:58.384024 10863 ts_manager.cc:194] Registered new tserver with Master: 72247991c7ae4381a4d7eee09020a200 (127.7.125.67:37989)
I20260808 14:28:58.384615 10863 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:53085
I20260808 14:28:58.390413 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 14:28:58 dist-test-slave-w206 krb5kdc[10814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199338, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 14:28:58 dist-test-slave-w206 krb5kdc[10814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199338, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:58.417464 10904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:58.409278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58770 (local address 127.7.125.126:42917)
0808 14:28:58.409471 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:58.409476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:58.409563 (+ 87us) server_negotiation.cc:407] Connection header received
0808 14:28:58.409684 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:58.409686 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:58.409731 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:58.409804 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:58.410505 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:58.411094 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:58.411898 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:58.412102 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:58.412177 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:58.414551 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:58.414554 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:58.414556 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:58.414581 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:58.415919 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:58.416323 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:58.416326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:58.416328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:58.416401 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:58.416636 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:58.416638 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:58.416639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:58.416866 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:58.416969 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:58.417161 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 14:28:58.417256 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260808 14:28:58.417943 10863 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58770
I20260808 14:28:58.419029 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 10927
I20260808 14:28:58.427474 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 11067
I20260808 14:28:58.436295 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 11209
I20260808 14:28:58.442855 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 10829
2026-08-08T14:28:58Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2263 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:28:58 dist-test-slave-w206 krb5kdc[11360](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:28:58 dist-test-slave-w206 krb5kdc[11360](info): set up 2 sockets
Aug 08 14:28:58 dist-test-slave-w206 krb5kdc[11360](info): commencing operation
krb5kdc: starting...
W20260808 14:28:59.507359 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:28:59 dist-test-slave-w206 krb5kdc[11360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:28:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:28:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:28:59.634600 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:44525
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:44525
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:28:59.746814 11373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:28:59.747177 11373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:28:59.747275 11373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:28:59.751200 11373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:28:59.751294 11373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:28:59.751332 11373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:28:59.751369 11373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:28:59.751402 11373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:28:59.756736 11373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:44525
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:44525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:28:59.757808 11373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:28:59.758872 11373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:28:59.767306 11381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:59.767313 11379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:28:59.767372 11378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:28:59.767632 11373 server_base.cc:1034] running on GCE node
I20260808 14:28:59.768172 11373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:28:59.769217 11373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:28:59.770380 11373 hybrid_clock.cc:648] HybridClock initialized: now 1786199339770360 us; error 31 us; skew 500 ppm
Aug 08 14:28:59 dist-test-slave-w206 krb5kdc[11360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199339, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:28:59.773120 11373 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:28:59.774042 11373 webserver.cc:460] Webserver started at http://127.7.125.126:38031/ using document root <none> and password file <none>
I20260808 14:28:59.774602 11373 fs_manager.cc:359] Metadata directory not provided
I20260808 14:28:59.774696 11373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:28:59.774927 11373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:28:59.776980 11373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "0660cdc959ea4715b6a4f4363346218a"
format_stamp: "Formatted at 2026-08-08 14:28:59 on dist-test-slave-w206"
server_key: "36e176f042b949ad4826caa0b30e8efc"
server_key_iv: "1b68870dc73bb942277f35e8dc349e54"
server_key_version: "encryptionkey@0"
I20260808 14:28:59.777581 11373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "0660cdc959ea4715b6a4f4363346218a"
format_stamp: "Formatted at 2026-08-08 14:28:59 on dist-test-slave-w206"
server_key: "36e176f042b949ad4826caa0b30e8efc"
server_key_iv: "1b68870dc73bb942277f35e8dc349e54"
server_key_version: "encryptionkey@0"
I20260808 14:28:59.781358 11373 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 14:28:59.783843 11388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:28:59.785036 11373 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:28:59.785185 11373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "0660cdc959ea4715b6a4f4363346218a"
format_stamp: "Formatted at 2026-08-08 14:28:59 on dist-test-slave-w206"
server_key: "36e176f042b949ad4826caa0b30e8efc"
server_key_iv: "1b68870dc73bb942277f35e8dc349e54"
server_key_version: "encryptionkey@0"
I20260808 14:28:59.785300 11373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:28:59.805377 11373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:28:59.808547 11373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:28:59.808764 11373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:28:59.816397 11373 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:44525
I20260808 14:28:59.816421 11442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:44525 every 8 connection(s)
I20260808 14:28:59.817384 11373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:28:59.820367 11443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:28:59.821657 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 11373
I20260808 14:28:59.821761 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:28:59.822063 7669 external_mini_cluster.cc:1428] Setting key 1ccb5cda68936387620ce08a9924a4d6
I20260808 14:28:59.826640 11443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0660cdc959ea4715b6a4f4363346218a: Bootstrap starting.
I20260808 14:28:59.828857 11443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0660cdc959ea4715b6a4f4363346218a: Neither blocks nor log segments found. Creating new log.
I20260808 14:28:59.829718 11443 log.cc:824] T 00000000000000000000000000000000 P 0660cdc959ea4715b6a4f4363346218a: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:28:59 dist-test-slave-w206 krb5kdc[11360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:28:59.831720 11443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0660cdc959ea4715b6a4f4363346218a: No bootstrap required, opened a new log
I20260808 14:28:59.835289 11443 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0660cdc959ea4715b6a4f4363346218a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0660cdc959ea4715b6a4f4363346218a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44525 } }
I20260808 14:28:59.835498 11443 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0660cdc959ea4715b6a4f4363346218a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:28:59.835532 11443 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0660cdc959ea4715b6a4f4363346218a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0660cdc959ea4715b6a4f4363346218a, State: Initialized, Role: FOLLOWER
I20260808 14:28:59.835965 11443 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0660cdc959ea4715b6a4f4363346218a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0660cdc959ea4715b6a4f4363346218a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44525 } }
I20260808 14:28:59.836046 11446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:28:59.823526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53712 (local address 127.7.125.126:44525)
0808 14:28:59.823977 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0808 14:28:59.823991 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 14:28:59.824027 (+ 36us) server_negotiation.cc:407] Connection header received
0808 14:28:59.824651 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:28:59.824667 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:28:59.824988 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:28:59.825326 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:28:59.826163 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:59.827302 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:59.828138 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:28:59.828623 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:28:59.828798 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:28:59.831358 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:28:59.831363 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:28:59.831376 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:28:59.831403 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:28:59.833105 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:59.833577 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:59.833582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:59.833586 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:59.833698 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:28:59.833986 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:28:59.833990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:28:59.833991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:28:59.834374 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:28:59.834600 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 14:28:59.834850 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 14:28:59.835165 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":144,"threads_started":1}
I20260808 14:28:59.836077 11443 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0660cdc959ea4715b6a4f4363346218a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:28:59.836362 11443 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0660cdc959ea4715b6a4f4363346218a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:28:59.836494 11443 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0660cdc959ea4715b6a4f4363346218a [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:28:59.837502 11443 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0660cdc959ea4715b6a4f4363346218a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0660cdc959ea4715b6a4f4363346218a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44525 } }
I20260808 14:28:59.837852 11443 leader_election.cc:302] T 00000000000000000000000000000000 P 0660cdc959ea4715b6a4f4363346218a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0660cdc959ea4715b6a4f4363346218a; no voters:
I20260808 14:28:59.838168 11443 leader_election.cc:288] T 00000000000000000000000000000000 P 0660cdc959ea4715b6a4f4363346218a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:28:59.838289 11448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0660cdc959ea4715b6a4f4363346218a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:28:59.838495 11448 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0660cdc959ea4715b6a4f4363346218a [term 1 LEADER]: Becoming Leader. State: Replica: 0660cdc959ea4715b6a4f4363346218a, State: Running, Role: LEADER
I20260808 14:28:59.838835 11448 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0660cdc959ea4715b6a4f4363346218a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0660cdc959ea4715b6a4f4363346218a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44525 } }
I20260808 14:28:59.839418 11443 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0660cdc959ea4715b6a4f4363346218a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:28:59.839955 11448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0660cdc959ea4715b6a4f4363346218a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0660cdc959ea4715b6a4f4363346218a. Latest consensus state: current_term: 1 leader_uuid: "0660cdc959ea4715b6a4f4363346218a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0660cdc959ea4715b6a4f4363346218a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44525 } } }
I20260808 14:28:59.840289 11448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0660cdc959ea4715b6a4f4363346218a [sys.catalog]: This master's current role is: LEADER
I20260808 14:28:59.840385 11449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0660cdc959ea4715b6a4f4363346218a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0660cdc959ea4715b6a4f4363346218a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0660cdc959ea4715b6a4f4363346218a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44525 } } }
I20260808 14:28:59.840519 11449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0660cdc959ea4715b6a4f4363346218a [sys.catalog]: This master's current role is: LEADER
I20260808 14:28:59.840782 11458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:28:59.843757 11458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:28:59.849344 11458 catalog_manager.cc:1347] Generated new cluster ID: 5437fa2630a6488d81e69dad54297c1a
I20260808 14:28:59.849423 11458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:28:59.873742 11458 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:28:59.875108 11458 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:28:59.884495 11458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0660cdc959ea4715b6a4f4363346218a: Generated new TSK 0
I20260808 14:28:59.885339 11458 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.125.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
I20260808 14:28:59.922961 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:44525
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:00.038306 11470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:00.038592 11470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:00.038669 11470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:00.043154 11470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:00.043238 11470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:00.043329 11470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:29:00.048545 11470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:44525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:00.049566 11470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:00.050588 11470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:00.060353 11479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:00.060463 11476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:00.060738 11470 server_base.cc:1034] running on GCE node
W20260808 14:29:00.060353 11477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:00.061269 11470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:00.061956 11470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:00.063139 11470 hybrid_clock.cc:648] HybridClock initialized: now 1786199340063113 us; error 47 us; skew 500 ppm
Aug 08 14:29:00 dist-test-slave-w206 krb5kdc[11360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199340, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:00.066097 11470 init.cc:377] Logged in from keytab as kudu/127.7.125.65@KRBTEST.COM (short username kudu)
I20260808 14:29:00.067051 11470 webserver.cc:460] Webserver started at http://127.7.125.65:38317/ using document root <none> and password file <none>
I20260808 14:29:00.067619 11470 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:00.067711 11470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:00.067957 11470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:00.070035 11470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "030ab6fbe305483aa4f1c6a6ad09b549"
format_stamp: "Formatted at 2026-08-08 14:29:00 on dist-test-slave-w206"
server_key: "f73795a62f60364157149f1b352e87d4"
server_key_iv: "7e3c112779058812b81b25491361dedc"
server_key_version: "encryptionkey@0"
I20260808 14:29:00.070668 11470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "030ab6fbe305483aa4f1c6a6ad09b549"
format_stamp: "Formatted at 2026-08-08 14:29:00 on dist-test-slave-w206"
server_key: "f73795a62f60364157149f1b352e87d4"
server_key_iv: "7e3c112779058812b81b25491361dedc"
server_key_version: "encryptionkey@0"
I20260808 14:29:00.074678 11470 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:29:00.077211 11486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:00.078513 11470 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:29:00.078758 11470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "030ab6fbe305483aa4f1c6a6ad09b549"
format_stamp: "Formatted at 2026-08-08 14:29:00 on dist-test-slave-w206"
server_key: "f73795a62f60364157149f1b352e87d4"
server_key_iv: "7e3c112779058812b81b25491361dedc"
server_key_version: "encryptionkey@0"
I20260808 14:29:00.078893 11470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:00.100723 11470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:00.103752 11470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:00.103979 11470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:00.104584 11470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:00.105434 11470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:00.105513 11470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:00.105584 11470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:00.105633 11470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:00.114742 11470 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:41729
I20260808 14:29:00.114766 11604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:41729 every 8 connection(s)
I20260808 14:29:00.115692 11470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:29:00.120765 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 11470
I20260808 14:29:00.120899 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:29:00.121201 7669 external_mini_cluster.cc:1428] Setting key dd1dbf8c054a1c6b7d3eb5311f04adfe
Aug 08 14:29:00 dist-test-slave-w206 krb5kdc[11360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199340, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:29:00.131105 11446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:00.117522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:55299 (local address 127.7.125.126:44525)
0808 14:29:00.117665 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:00.117669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:00.118495 (+ 826us) server_negotiation.cc:407] Connection header received
0808 14:29:00.119983 (+ 1488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:00.119986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:00.120046 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:00.120127 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:00.121425 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:00.122017 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:00.123345 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:00.123768 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:00.123898 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:00.126730 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:00.126733 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:00.126735 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:00.126759 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:00.128562 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:00.129191 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:00.129195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:00.129198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:00.129307 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:00.129733 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:00.129735 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:00.129737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:00.129924 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:00.130005 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:00.130699 (+ 694us) server_negotiation.cc:299] Negotiation successful
0808 14:29:00.130908 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 14:29:00.132206 11608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:00.117897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:44525 (local address 127.7.125.65:55299)
0808 14:29:00.118340 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:00.118383 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:00.119746 (+ 1363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:00.120284 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:00.120294 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:00.120646 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:00.121224 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:00.121277 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:00.122163 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:00.122166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:00.123217 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:00.123227 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:00.123917 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:00.123920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:00.124054 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:00.124670 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:00.124686 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:00.126555 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:00.128698 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:00.128703 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:00.128716 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:00.129081 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:00.129424 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:00.129431 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:00.129436 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:00.129634 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:00.130041 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:00.130050 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:00.130412 (+ 362us) client_negotiation.cc:769] Sending connection context
0808 14:29:00.130657 (+ 245us) client_negotiation.cc:240] Negotiation successful
0808 14:29:00.131081 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":112,"threads_started":1}
I20260808 14:29:00.133579 11606 heartbeater.cc:347] Connected to a master server at 127.7.125.126:44525
WARNING: no policy specified for kudu/127.7.125.66@KRBTEST.COM; defaulting to no policy
I20260808 14:29:00.133838 11606 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:00.134337 11606 heartbeater.cc:511] Master 127.7.125.126:44525 requested a full tablet report, sending...
I20260808 14:29:00.135972 11407 ts_manager.cc:194] Registered new tserver with Master: 030ab6fbe305483aa4f1c6a6ad09b549 (127.7.125.65:41729)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.66@KRBTEST.COM" created.
I20260808 14:29:00.137177 11407 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:55299
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
I20260808 14:29:00.155077 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:44525
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:00.270349 11611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:00.270648 11611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:00.270728 11611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:00.274842 11611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:00.274930 11611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:00.275053 11611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:29:00.280434 11611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:44525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:00.281594 11611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:00.282686 11611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:00.292660 11619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:00.292723 11617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:00.292691 11616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:00.292845 11611 server_base.cc:1034] running on GCE node
I20260808 14:29:00.293355 11611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:00.294090 11611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:00.295279 11611 hybrid_clock.cc:648] HybridClock initialized: now 1786199340295257 us; error 36 us; skew 500 ppm
Aug 08 14:29:00 dist-test-slave-w206 krb5kdc[11360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199340, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:00.298090 11611 init.cc:377] Logged in from keytab as kudu/127.7.125.66@KRBTEST.COM (short username kudu)
I20260808 14:29:00.299093 11611 webserver.cc:460] Webserver started at http://127.7.125.66:36577/ using document root <none> and password file <none>
I20260808 14:29:00.299662 11611 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:00.299754 11611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:00.299988 11611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:00.302114 11611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "0da1e7ef0502462a879e921d338595c6"
format_stamp: "Formatted at 2026-08-08 14:29:00 on dist-test-slave-w206"
server_key: "d01f34aa8f61140c49d91ad3ac6f59a8"
server_key_iv: "cb81ede581b68b8ae7020d88eb17e616"
server_key_version: "encryptionkey@0"
I20260808 14:29:00.302695 11611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "0da1e7ef0502462a879e921d338595c6"
format_stamp: "Formatted at 2026-08-08 14:29:00 on dist-test-slave-w206"
server_key: "d01f34aa8f61140c49d91ad3ac6f59a8"
server_key_iv: "cb81ede581b68b8ae7020d88eb17e616"
server_key_version: "encryptionkey@0"
I20260808 14:29:00.306558 11611 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 14:29:00.309108 11626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:00.310420 11611 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 14:29:00.310559 11611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "0da1e7ef0502462a879e921d338595c6"
format_stamp: "Formatted at 2026-08-08 14:29:00 on dist-test-slave-w206"
server_key: "d01f34aa8f61140c49d91ad3ac6f59a8"
server_key_iv: "cb81ede581b68b8ae7020d88eb17e616"
server_key_version: "encryptionkey@0"
I20260808 14:29:00.310670 11611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:00.334036 11611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:00.337443 11611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:00.337659 11611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:00.338277 11611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:00.339179 11611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:00.339254 11611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:00.339341 11611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:00.339388 11611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:00.348743 11611 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:34771
I20260808 14:29:00.348767 11742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:34771 every 8 connection(s)
I20260808 14:29:00.349658 11611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:29:00.352514 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 11611
I20260808 14:29:00.352639 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:29:00.353017 7669 external_mini_cluster.cc:1428] Setting key fa351e80a54b3e2663f330f986457382
Aug 08 14:29:00 dist-test-slave-w206 krb5kdc[11360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199340, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.125.67@KRBTEST.COM; defaulting to no policy
I20260808 14:29:00.365551 11446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:00.351558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:51479 (local address 127.7.125.126:44525)
0808 14:29:00.351701 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:00.351705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:00.352787 (+ 1082us) server_negotiation.cc:407] Connection header received
0808 14:29:00.354476 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:00.354479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:00.354544 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:00.354611 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:00.355871 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:00.356709 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:00.358580 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:00.358791 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:00.358870 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:00.361783 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:00.361786 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:00.361788 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:00.361811 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:00.363422 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:00.363950 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:00.363954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:00.363956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:00.364041 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:00.364424 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:00.364427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:00.364429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:00.364615 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:00.364726 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:00.365233 (+ 507us) server_negotiation.cc:299] Negotiation successful
0808 14:29:00.365362 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 14:29:00.366330 11745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:00.351956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:44525 (local address 127.7.125.66:51479)
0808 14:29:00.352581 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:00.352648 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:00.354234 (+ 1586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:00.354808 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:00.354814 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:00.355253 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:00.355727 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:00.355740 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:00.356881 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:00.356887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:00.358457 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:00.358472 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:00.358945 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:00.358952 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:00.359230 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:00.359884 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:00.359912 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:00.361610 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:00.363550 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:00.363556 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:00.363572 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:00.363835 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:00.364137 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:00.364141 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:00.364143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:00.364324 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:00.364724 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:00.364732 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:00.364989 (+ 257us) client_negotiation.cc:769] Sending connection context
0808 14:29:00.365178 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 14:29:00.365507 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":111,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.67@KRBTEST.COM" created.
I20260808 14:29:00.367681 11743 heartbeater.cc:347] Connected to a master server at 127.7.125.126:44525
I20260808 14:29:00.367931 11743 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:00.368463 11743 heartbeater.cc:511] Master 127.7.125.126:44525 requested a full tablet report, sending...
I20260808 14:29:00.369632 11407 ts_manager.cc:194] Registered new tserver with Master: 0da1e7ef0502462a879e921d338595c6 (127.7.125.66:34771)
I20260808 14:29:00.370205 11407 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:51479
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
I20260808 14:29:00.386050 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:44525
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:00.499567 11748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:00.499825 11748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:00.499871 11748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:00.503970 11748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:00.504024 11748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:00.504083 11748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:29:00.509428 11748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:44525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:00.510488 11748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:00.511480 11748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:29:00.521468 11748 server_base.cc:1034] running on GCE node
W20260808 14:29:00.521494 11754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:00.521462 11757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:00.521494 11753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:00.522135 11748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:00.522820 11748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:00.524041 11748 hybrid_clock.cc:648] HybridClock initialized: now 1786199340524031 us; error 30 us; skew 500 ppm
Aug 08 14:29:00 dist-test-slave-w206 krb5kdc[11360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199340, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:00.526897 11748 init.cc:377] Logged in from keytab as kudu/127.7.125.67@KRBTEST.COM (short username kudu)
I20260808 14:29:00.527824 11748 webserver.cc:460] Webserver started at http://127.7.125.67:43653/ using document root <none> and password file <none>
I20260808 14:29:00.528383 11748 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:00.528473 11748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:00.528703 11748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:00.530723 11748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "4777aa1f2aac46ecb8a8396b52abc372"
format_stamp: "Formatted at 2026-08-08 14:29:00 on dist-test-slave-w206"
server_key: "6ff7c4f19edf1324d13dba5db9e00e8e"
server_key_iv: "5dadbf866f6b11a0ca855051941b324d"
server_key_version: "encryptionkey@0"
I20260808 14:29:00.531324 11748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "4777aa1f2aac46ecb8a8396b52abc372"
format_stamp: "Formatted at 2026-08-08 14:29:00 on dist-test-slave-w206"
server_key: "6ff7c4f19edf1324d13dba5db9e00e8e"
server_key_iv: "5dadbf866f6b11a0ca855051941b324d"
server_key_version: "encryptionkey@0"
I20260808 14:29:00.534924 11748 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:29:00.537247 11766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:00.538446 11748 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 14:29:00.538578 11748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "4777aa1f2aac46ecb8a8396b52abc372"
format_stamp: "Formatted at 2026-08-08 14:29:00 on dist-test-slave-w206"
server_key: "6ff7c4f19edf1324d13dba5db9e00e8e"
server_key_iv: "5dadbf866f6b11a0ca855051941b324d"
server_key_version: "encryptionkey@0"
I20260808 14:29:00.538686 11748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:00.571718 11748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:00.575155 11748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:00.575362 11748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:00.575994 11748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:00.576903 11748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:00.576973 11748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:00.577040 11748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:00.577090 11748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:00.586371 11748 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:46833
I20260808 14:29:00.586402 11884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:46833 every 8 connection(s)
I20260808 14:29:00.587319 11748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:29:00.593839 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 11748
I20260808 14:29:00.594005 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:29:00.594376 7669 external_mini_cluster.cc:1428] Setting key 45ddeedbb4f5390efb17907793ca24a4
Aug 08 14:29:00 dist-test-slave-w206 krb5kdc[11360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199340, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:29:00.601858 11446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:00.589203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:53503 (local address 127.7.125.126:44525)
0808 14:29:00.589347 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:00.589351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:00.590233 (+ 882us) server_negotiation.cc:407] Connection header received
0808 14:29:00.591827 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:00.591831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:00.591894 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:00.592041 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:00.593302 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:00.594154 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:00.595577 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:00.595858 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:00.595976 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:00.598462 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:00.598465 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:00.598467 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:00.598493 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:00.599954 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:00.600500 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:00.600503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:00.600505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:00.600604 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:00.600926 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:00.600928 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:00.600930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:00.601104 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:00.601178 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:00.601634 (+ 456us) server_negotiation.cc:299] Negotiation successful
0808 14:29:00.601729 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 14:29:00.602715 11887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:00.589591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:44525 (local address 127.7.125.67:53503)
0808 14:29:00.590056 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:00.590129 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:00.591475 (+ 1346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:00.592228 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:00.592244 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:00.592631 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:00.593143 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:00.593158 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:00.594303 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:00.594306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:00.595447 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:00.595456 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:00.596006 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:00.596009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:00.596147 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:00.596763 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:00.596779 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:00.598299 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:00.600076 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:00.600081 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:00.600093 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:00.600361 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:00.600686 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:00.600689 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:00.600690 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:00.600844 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:00.601190 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:00.601195 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:00.601428 (+ 233us) client_negotiation.cc:769] Sending connection context
0808 14:29:00.601587 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 14:29:00.601858 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":108,"threads_started":1}
I20260808 14:29:00.603924 11885 heartbeater.cc:347] Connected to a master server at 127.7.125.126:44525
I20260808 14:29:00.604175 11885 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:00.604676 11885 heartbeater.cc:511] Master 127.7.125.126:44525 requested a full tablet report, sending...
I20260808 14:29:00.605711 11407 ts_manager.cc:194] Registered new tserver with Master: 4777aa1f2aac46ecb8a8396b52abc372 (127.7.125.67:46833)
I20260808 14:29:00.606240 11407 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:53503
I20260808 14:29:00.608511 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 14:29:00 dist-test-slave-w206 krb5kdc[11360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 14:29:00 dist-test-slave-w206 krb5kdc[11360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.65@KRBTEST.COM
I20260808 14:29:00.635929 11892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:00.626497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50246 (local address 127.7.125.65:41729)
0808 14:29:00.626783 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:00.626799 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:00.626827 (+ 28us) server_negotiation.cc:407] Connection header received
0808 14:29:00.626880 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:00.626885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:00.627141 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:00.627269 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:00.627999 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:00.628771 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:00.629600 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:00.629953 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:00.630035 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:00.632652 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:00.632662 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:00.632676 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:00.632700 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:00.634279 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:00.634694 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:00.634699 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:00.634701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:00.634782 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:00.635073 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:00.635075 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:00.635076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:00.635320 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:00.635490 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:00.635656 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 14:29:00.635786 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":81,"threads_started":1}
Aug 08 14:29:00 dist-test-slave-w206 krb5kdc[11360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:29:00.647516 11446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:00.638863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53720 (local address 127.7.125.126:44525)
0808 14:29:00.639044 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:00.639051 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:00.639068 (+ 17us) server_negotiation.cc:407] Connection header received
0808 14:29:00.639219 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:00.639222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:00.639275 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:00.639361 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:00.640087 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:00.640748 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:00.641651 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:00.641891 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:00.642002 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:00.644492 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:00.644495 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:00.644498 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:00.644526 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:00.645768 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:00.646178 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:00.646182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:00.646184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:00.646272 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:00.646553 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:00.646557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:00.646559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:00.646783 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:00.646947 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:00.647213 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 14:29:00.647338 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 14:29:00.647934 11407 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53720
I20260808 14:29:00.648590 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 11470
I20260808 14:29:00.656212 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 11611
I20260808 14:29:00.662602 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 11748
I20260808 14:29:00.668182 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 11373
2026-08-08T14:29:00Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2225 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:29:00 dist-test-slave-w206 krb5kdc[11898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:29:00 dist-test-slave-w206 krb5kdc[11898](info): set up 2 sockets
Aug 08 14:29:00 dist-test-slave-w206 krb5kdc[11898](info): commencing operation
krb5kdc: starting...
W20260808 14:29:01.835642 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:29:01 dist-test-slave-w206 krb5kdc[11898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:29:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:29:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:29:01.965792 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:33635
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:36021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:33635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:02.080583 11911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:02.080878 11911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:02.080966 11911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:02.085263 11911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:29:02.085337 11911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:02.085357 11911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:02.085376 11911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:29:02.085391 11911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:29:02.090813 11911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:36021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:33635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:33635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:02.092023 11911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:02.093075 11911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:02.101449 11920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:02.101445 11918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:02.101550 11916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:02.101682 11911 server_base.cc:1034] running on GCE node
I20260808 14:29:02.102289 11911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:02.103461 11911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:02.104628 11911 hybrid_clock.cc:648] HybridClock initialized: now 1786199342104606 us; error 35 us; skew 500 ppm
Aug 08 14:29:02 dist-test-slave-w206 krb5kdc[11898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199342, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:02.107522 11911 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:29:02.108253 11911 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:02.108356 11911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:02.108595 11911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:02.110653 11911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "5abf19171918479b85311c66044e83cb"
format_stamp: "Formatted at 2026-08-08 14:29:02 on dist-test-slave-w206"
server_key: "188e4b00b311cb3992056e4ae18dd09c"
server_key_iv: "7a03deb4fea2bc5bd7cb9c3da22e0225"
server_key_version: "encryptionkey@0"
I20260808 14:29:02.111274 11911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "5abf19171918479b85311c66044e83cb"
format_stamp: "Formatted at 2026-08-08 14:29:02 on dist-test-slave-w206"
server_key: "188e4b00b311cb3992056e4ae18dd09c"
server_key_iv: "7a03deb4fea2bc5bd7cb9c3da22e0225"
server_key_version: "encryptionkey@0"
I20260808 14:29:02.115083 11911 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 14:29:02.117486 11926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:02.118770 11911 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:29:02.118932 11911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "5abf19171918479b85311c66044e83cb"
format_stamp: "Formatted at 2026-08-08 14:29:02 on dist-test-slave-w206"
server_key: "188e4b00b311cb3992056e4ae18dd09c"
server_key_iv: "7a03deb4fea2bc5bd7cb9c3da22e0225"
server_key_version: "encryptionkey@0"
I20260808 14:29:02.119102 11911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:02.153147 11911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:02.156723 11911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:02.156955 11911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:02.164543 11911 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:33635
I20260808 14:29:02.164829 11980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:33635 every 8 connection(s)
I20260808 14:29:02.165293 11911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:29:02.168182 11981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:02.173146 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 11911
I20260808 14:29:02.173313 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:29:02.173609 7669 external_mini_cluster.cc:1428] Setting key 32a4612a993be113b82f4460cba7fab6
I20260808 14:29:02.174340 11981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb: Bootstrap starting.
I20260808 14:29:02.176877 11981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:02.177695 11981 log.cc:824] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:02.179811 11981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb: No bootstrap required, opened a new log
Aug 08 14:29:02 dist-test-slave-w206 krb5kdc[11898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:29:02.183905 11981 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5abf19171918479b85311c66044e83cb" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33635 } }
I20260808 14:29:02.184177 11981 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:02.184264 11981 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5abf19171918479b85311c66044e83cb, State: Initialized, Role: FOLLOWER
I20260808 14:29:02.184767 11981 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5abf19171918479b85311c66044e83cb" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33635 } }
I20260808 14:29:02.184897 11981 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:29:02.184986 11981 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:29:02.185111 11981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:02.186297 11981 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5abf19171918479b85311c66044e83cb" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33635 } }
I20260808 14:29:02.186687 11981 leader_election.cc:302] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5abf19171918479b85311c66044e83cb; no voters:
I20260808 14:29:02.187033 11981 leader_election.cc:288] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:29:02.187143 11986 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:29:02.187418 11986 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb [term 1 LEADER]: Becoming Leader. State: Replica: 5abf19171918479b85311c66044e83cb, State: Running, Role: LEADER
I20260808 14:29:02.187793 11986 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5abf19171918479b85311c66044e83cb" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33635 } }
I20260808 14:29:02.188447 11981 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:29:02.188937 11984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:02.175115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36800 (local address 127.7.125.126:33635)
0808 14:29:02.175577 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:02.175592 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:02.175630 (+ 38us) server_negotiation.cc:407] Connection header received
0808 14:29:02.176290 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:02.176307 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:02.176659 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:02.177006 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:02.177831 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:02.178827 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:02.179741 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:02.180295 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:02.180473 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:02.183383 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:02.183388 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:02.183402 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:02.183428 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:02.185447 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:02.185923 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:02.185927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:02.185932 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:02.186040 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:02.186429 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:02.186435 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:02.186439 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:02.186981 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:02.187291 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:02.187633 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 14:29:02.187954 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":147,"threads_started":1}
I20260808 14:29:02.189592 11987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5abf19171918479b85311c66044e83cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5abf19171918479b85311c66044e83cb" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33635 } } }
I20260808 14:29:02.189560 11988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5abf19171918479b85311c66044e83cb. Latest consensus state: current_term: 1 leader_uuid: "5abf19171918479b85311c66044e83cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5abf19171918479b85311c66044e83cb" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33635 } } }
I20260808 14:29:02.189703 11988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:02.189714 11987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb [sys.catalog]: This master's current role is: LEADER
W20260808 14:29:02.193595 12001 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:29:02.193687 12001 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:29:02.193799 11995 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:29:02.194689 11995 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:29:02.201387 11995 catalog_manager.cc:1347] Generated new cluster ID: 4b1cccf6a19248e2adc868d78b17a40d
I20260808 14:29:02.201453 11995 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:29:02.230427 11995 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:29:02.232110 11995 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:29:02.239430 11995 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb: Generated new TSK 0
I20260808 14:29:02.240214 11995 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.125.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
I20260808 14:29:02.283816 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:33635
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:36021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:02.416546 12007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:02.416813 12007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:02.416862 12007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:02.421048 12007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:02.421118 12007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:02.421224 12007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:29:02.426888 12007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:36021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:33635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:02.428061 12007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:02.429134 12007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:02.438915 12013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:02.438884 12015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:02.439198 12007 server_base.cc:1034] running on GCE node
W20260808 14:29:02.439428 12012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:02.439893 12007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:02.440653 12007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:02.441861 12007 hybrid_clock.cc:648] HybridClock initialized: now 1786199342441826 us; error 52 us; skew 500 ppm
Aug 08 14:29:02 dist-test-slave-w206 krb5kdc[11898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199342, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:02.444904 12007 init.cc:377] Logged in from keytab as kudu/127.7.125.65@KRBTEST.COM (short username kudu)
I20260808 14:29:02.445688 12007 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:02.445794 12007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:02.446130 12007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:02.448365 12007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "72383143a2504ac1aa63010aa84fa972"
format_stamp: "Formatted at 2026-08-08 14:29:02 on dist-test-slave-w206"
server_key: "30df4b3c31671e67133d43997a9947fb"
server_key_iv: "95fe734182296e6608226425fa5b5753"
server_key_version: "encryptionkey@0"
I20260808 14:29:02.448974 12007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "72383143a2504ac1aa63010aa84fa972"
format_stamp: "Formatted at 2026-08-08 14:29:02 on dist-test-slave-w206"
server_key: "30df4b3c31671e67133d43997a9947fb"
server_key_iv: "95fe734182296e6608226425fa5b5753"
server_key_version: "encryptionkey@0"
I20260808 14:29:02.453004 12007 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:29:02.455626 12021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:02.456871 12007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:29:02.457021 12007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "72383143a2504ac1aa63010aa84fa972"
format_stamp: "Formatted at 2026-08-08 14:29:02 on dist-test-slave-w206"
server_key: "30df4b3c31671e67133d43997a9947fb"
server_key_iv: "95fe734182296e6608226425fa5b5753"
server_key_version: "encryptionkey@0"
I20260808 14:29:02.457134 12007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:02.473120 12007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:02.476342 12007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:02.476548 12007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:02.477154 12007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:02.477990 12007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:02.478061 12007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:02.478133 12007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:02.478186 12007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:02.487349 12007 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:37795
I20260808 14:29:02.487367 12143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:37795 every 8 connection(s)
I20260808 14:29:02.487994 12007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:29:02.492465 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 12007
I20260808 14:29:02.492604 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:29:02.492900 7669 external_mini_cluster.cc:1428] Setting key 1af561161b4d344d391769b350b36dd1
Aug 08 14:29:02 dist-test-slave-w206 krb5kdc[11898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199342, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.125.66@KRBTEST.COM; defaulting to no policy
I20260808 14:29:02.503413 11984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:02.489838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:54143 (local address 127.7.125.126:33635)
0808 14:29:02.490013 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:02.490016 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:02.490962 (+ 946us) server_negotiation.cc:407] Connection header received
0808 14:29:02.492441 (+ 1479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:02.492445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:02.492509 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:02.492638 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:02.494248 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:02.494816 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:02.496136 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:02.496400 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:02.496523 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:02.499323 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:02.499327 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:02.499329 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:02.499356 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:02.501015 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:02.501619 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:02.501623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:02.501626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:02.501718 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:02.502112 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:02.502116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:02.502117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:02.502322 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:02.502441 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:02.503072 (+ 631us) server_negotiation.cc:299] Negotiation successful
0808 14:29:02.503215 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:29:02.504457 12146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:02.490264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:33635 (local address 127.7.125.65:54143)
0808 14:29:02.490776 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:02.490824 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:02.492163 (+ 1339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:02.492868 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:02.492877 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:02.493344 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:02.493963 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:02.493979 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:02.494960 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:02.494971 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:02.495993 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:02.496003 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:02.496570 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:02.496582 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:02.496767 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:02.497470 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:02.497488 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:02.499128 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:02.501168 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:02.501175 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:02.501191 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:02.501494 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:02.501827 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:02.501832 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:02.501834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:02.501993 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:02.502441 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:02.502450 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:02.502779 (+ 329us) client_negotiation.cc:769] Sending connection context
0808 14:29:02.502984 (+ 205us) client_negotiation.cc:240] Negotiation successful
0808 14:29:02.503425 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":120,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.66@KRBTEST.COM" created.
I20260808 14:29:02.505991 12144 heartbeater.cc:347] Connected to a master server at 127.7.125.126:33635
I20260808 14:29:02.506254 12144 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:02.506762 12144 heartbeater.cc:511] Master 127.7.125.126:33635 requested a full tablet report, sending...
I20260808 14:29:02.508585 11944 ts_manager.cc:194] Registered new tserver with Master: 72383143a2504ac1aa63010aa84fa972 (127.7.125.65:37795)
I20260808 14:29:02.510296 11944 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:54143
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
I20260808 14:29:02.523548 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:33635
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:36021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:02.637472 12149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:02.637730 12149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:02.637807 12149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:02.642373 12149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:02.642452 12149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:02.642562 12149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:29:02.648264 12149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:36021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:33635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:02.649389 12149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:02.650432 12149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:02.660035 12155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:02.660241 12149 server_base.cc:1034] running on GCE node
W20260808 14:29:02.660066 12156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:02.660382 12158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:02.660849 12149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:02.661616 12149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:02.662859 12149 hybrid_clock.cc:648] HybridClock initialized: now 1786199342662838 us; error 32 us; skew 500 ppm
Aug 08 14:29:02 dist-test-slave-w206 krb5kdc[11898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199342, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:02.670519 12149 init.cc:377] Logged in from keytab as kudu/127.7.125.66@KRBTEST.COM (short username kudu)
I20260808 14:29:02.671342 12149 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:02.671411 12149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:02.671576 12149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:02.673563 12149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "8e0fec4a16a34ccab687363caadd063d"
format_stamp: "Formatted at 2026-08-08 14:29:02 on dist-test-slave-w206"
server_key: "ef21c7a100feb726a8b1db3a3a1b7912"
server_key_iv: "f5d639f54e9eacb6cf5aee4a7cfc4eac"
server_key_version: "encryptionkey@0"
I20260808 14:29:02.674070 12149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "8e0fec4a16a34ccab687363caadd063d"
format_stamp: "Formatted at 2026-08-08 14:29:02 on dist-test-slave-w206"
server_key: "ef21c7a100feb726a8b1db3a3a1b7912"
server_key_iv: "f5d639f54e9eacb6cf5aee4a7cfc4eac"
server_key_version: "encryptionkey@0"
I20260808 14:29:02.677747 12149 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 14:29:02.680248 12164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:02.681463 12149 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:29:02.681589 12149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "8e0fec4a16a34ccab687363caadd063d"
format_stamp: "Formatted at 2026-08-08 14:29:02 on dist-test-slave-w206"
server_key: "ef21c7a100feb726a8b1db3a3a1b7912"
server_key_iv: "f5d639f54e9eacb6cf5aee4a7cfc4eac"
server_key_version: "encryptionkey@0"
I20260808 14:29:02.681694 12149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:02.697235 12149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:02.700517 12149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:02.700714 12149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:02.701295 12149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:02.702107 12149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:02.702159 12149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:02.702239 12149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:02.702278 12149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:02.711452 12149 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:44007
I20260808 14:29:02.711481 12283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:44007 every 8 connection(s)
I20260808 14:29:02.712081 12149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:29:02.720844 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 12149
I20260808 14:29:02.720983 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:29:02.721372 7669 external_mini_cluster.cc:1428] Setting key c50bed8b2ad49d0c829bf11010315338
Aug 08 14:29:02 dist-test-slave-w206 krb5kdc[11898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199342, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:29:02.726583 11984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:02.713921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:36327 (local address 127.7.125.126:33635)
0808 14:29:02.714061 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:02.714072 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:02.714832 (+ 760us) server_negotiation.cc:407] Connection header received
0808 14:29:02.716237 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:02.716240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:02.716301 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:02.716411 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:02.717571 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:02.718110 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:02.719318 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:02.719538 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:02.719651 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:02.722537 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:02.722541 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:02.722544 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:02.722571 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:02.724312 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:02.724889 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:02.724894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:02.724896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:02.725030 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:02.725411 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:02.725418 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:02.725421 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:02.725684 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:02.725805 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:02.726287 (+ 482us) server_negotiation.cc:299] Negotiation successful
0808 14:29:02.726435 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 14:29:02.727421 12286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:02.714201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:33635 (local address 127.7.125.66:36327)
0808 14:29:02.714662 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:02.714704 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:02.716013 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:02.716596 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:02.716603 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:02.716966 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:02.717427 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:02.717439 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:02.718239 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:02.718243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:02.719192 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:02.719202 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:02.719696 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:02.719701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:02.719871 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:02.720500 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:02.720517 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:02.722354 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:02.724464 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:02.724470 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:02.724486 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:02.724775 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:02.725158 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:02.725160 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:02.725162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:02.725307 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:02.725812 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:02.725817 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:02.726043 (+ 226us) client_negotiation.cc:769] Sending connection context
0808 14:29:02.726196 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 14:29:02.726456 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":118,"threads_started":1}
I20260808 14:29:02.728619 12284 heartbeater.cc:347] Connected to a master server at 127.7.125.126:33635
I20260808 14:29:02.728907 12284 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:02.729442 12284 heartbeater.cc:511] Master 127.7.125.126:33635 requested a full tablet report, sending...
I20260808 14:29:02.730468 11944 ts_manager.cc:194] Registered new tserver with Master: 8e0fec4a16a34ccab687363caadd063d (127.7.125.66:44007)
I20260808 14:29:02.731109 11944 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:36327
WARNING: no policy specified for kudu/127.7.125.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
I20260808 14:29:02.753925 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:33635
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:36021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:02.869149 12290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:02.869415 12290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:02.869501 12290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:02.873726 12290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:02.873795 12290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:02.873924 12290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:29:02.879299 12290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:36021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:33635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:02.880443 12290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:02.881496 12290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:02.891366 12298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:02.891472 12290 server_base.cc:1034] running on GCE node
W20260808 14:29:02.891362 12296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:02.891336 12295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:02.892145 12290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:02.892886 12290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:02.894105 12290 hybrid_clock.cc:648] HybridClock initialized: now 1786199342894092 us; error 28 us; skew 500 ppm
Aug 08 14:29:02 dist-test-slave-w206 krb5kdc[11898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199342, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:02.897267 12290 init.cc:377] Logged in from keytab as kudu/127.7.125.67@KRBTEST.COM (short username kudu)
I20260808 14:29:02.898030 12290 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:02.898100 12290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:02.898277 12290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:02.900559 12290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "25a4d431a271489ea414b4df52509cf6"
format_stamp: "Formatted at 2026-08-08 14:29:02 on dist-test-slave-w206"
server_key: "c9e5089a410b957760c7acebebdd92e2"
server_key_iv: "3ad05fe29dd43e8a8f8feede416d02b0"
server_key_version: "encryptionkey@0"
I20260808 14:29:02.901114 12290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "25a4d431a271489ea414b4df52509cf6"
format_stamp: "Formatted at 2026-08-08 14:29:02 on dist-test-slave-w206"
server_key: "c9e5089a410b957760c7acebebdd92e2"
server_key_iv: "3ad05fe29dd43e8a8f8feede416d02b0"
server_key_version: "encryptionkey@0"
I20260808 14:29:02.905076 12290 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:29:02.907564 12304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:02.908914 12290 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:29:02.909046 12290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "25a4d431a271489ea414b4df52509cf6"
format_stamp: "Formatted at 2026-08-08 14:29:02 on dist-test-slave-w206"
server_key: "c9e5089a410b957760c7acebebdd92e2"
server_key_iv: "3ad05fe29dd43e8a8f8feede416d02b0"
server_key_version: "encryptionkey@0"
I20260808 14:29:02.909159 12290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:02.932356 12290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:02.935832 12290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:02.936069 12290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:02.936751 12290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:02.937680 12290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:02.937732 12290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:02.937808 12290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:02.937851 12290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:02.948081 12290 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:40191
I20260808 14:29:02.948097 12423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:40191 every 8 connection(s)
I20260808 14:29:02.948768 12290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:29:02.950899 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 12290
I20260808 14:29:02.951078 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:29:02.951392 7669 external_mini_cluster.cc:1428] Setting key e3cf22b06b21bf5d4aed86c1c1f7b8c8
Aug 08 14:29:02 dist-test-slave-w206 krb5kdc[11898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199342, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:29:02.963080 11984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:02.950591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:32917 (local address 127.7.125.126:33635)
0808 14:29:02.950775 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:02.950781 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:02.951885 (+ 1104us) server_negotiation.cc:407] Connection header received
0808 14:29:02.953371 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:02.953375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:02.953429 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:02.953518 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:02.954772 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:02.955380 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:02.956558 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:02.956762 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:02.956846 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:02.959427 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:02.959431 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:02.959433 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:02.959457 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:02.960968 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:02.961604 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:02.961607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:02.961608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:02.961687 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:02.962010 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:02.962012 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:02.962013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:02.962179 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:02.962253 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:02.962730 (+ 477us) server_negotiation.cc:299] Negotiation successful
0808 14:29:02.962866 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 14:29:02.963982 12426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:02.951122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:33635 (local address 127.7.125.67:32917)
0808 14:29:02.951701 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:02.951760 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:02.953158 (+ 1398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:02.953720 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:02.953730 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:02.954127 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:02.954642 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:02.954655 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:02.955498 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:02.955501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:02.956446 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:02.956455 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:02.956903 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:02.956906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:02.957054 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:02.957633 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:02.957651 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:02.959251 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:02.961068 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:02.961074 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:02.961085 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:02.961467 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:02.961773 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:02.961775 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:02.961777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:02.961927 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:02.962269 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:02.962274 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:02.962523 (+ 249us) client_negotiation.cc:769] Sending connection context
0808 14:29:02.962687 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 14:29:02.962949 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":146,"threads_started":1}
I20260808 14:29:02.965329 12424 heartbeater.cc:347] Connected to a master server at 127.7.125.126:33635
I20260808 14:29:02.965591 12424 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:02.966061 12424 heartbeater.cc:511] Master 127.7.125.126:33635 requested a full tablet report, sending...
I20260808 14:29:02.967139 11943 ts_manager.cc:194] Registered new tserver with Master: 25a4d431a271489ea414b4df52509cf6 (127.7.125.67:40191)
I20260808 14:29:02.967765 11943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:32917
I20260808 14:29:02.977577 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:29:02.987970 11984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:02.979691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36816 (local address 127.7.125.126:33635)
0808 14:29:02.979874 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:02.979879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:02.980041 (+ 162us) server_negotiation.cc:407] Connection header received
0808 14:29:02.980199 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:02.980202 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:02.980248 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:02.980338 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:02.981076 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:02.981605 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:02.982454 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:02.982688 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:02.982793 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:02.984264 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:02.984268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:02.984270 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:02.984295 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:02.985777 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:02.986286 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:02.986288 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:02.986290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:02.986373 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:02.986933 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:02.986937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:02.986939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:02.987281 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:02.987432 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:02.987679 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 14:29:02.987798 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260808 14:29:02.991107 11943 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:29:02.994164 11943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:29:03.008922 12440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:03.003477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:40191 (local address 127.0.0.1:51692)
0808 14:29:03.004005 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:03.004022 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:03.004139 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:03.004781 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:03.004784 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:03.004802 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:03.004897 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:03.004905 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:03.006318 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:03.006322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:03.007451 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:03.007459 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:03.008337 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:03.008341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:03.008447 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:03.008483 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 14:29:03.008591 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 14:29:03.008733 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":26,"thread_start_us":84,"threads_started":1}
I20260808 14:29:03.009004 12441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:03.003909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51692 (local address 127.7.125.67:40191)
0808 14:29:03.004359 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:03.004376 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:03.004401 (+ 25us) server_negotiation.cc:407] Connection header received
0808 14:29:03.004478 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:03.004485 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:03.004645 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:03.004804 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:03.005196 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:03.006160 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:03.007566 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:03.008196 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:03.008336 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:03.008484 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:03.008733 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 14:29:03.008834 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":145,"threads_started":1}
I20260808 14:29:03.009230 12437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:03.002730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41554 (local address 127.7.125.65:37795)
0808 14:29:03.003158 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:03.003167 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:03.003192 (+ 25us) server_negotiation.cc:407] Connection header received
0808 14:29:03.003507 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:03.003513 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:03.003729 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:03.003891 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:03.004380 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:03.005556 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:03.007258 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:03.008194 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:03.008323 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:03.008486 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:03.008964 (+ 478us) server_negotiation.cc:299] Negotiation successful
0808 14:29:03.009072 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":87,"threads_started":1}
I20260808 14:29:03.009588 12436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:03.002594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:37795 (local address 127.0.0.1:41554)
0808 14:29:03.003062 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:03.003116 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:03.003246 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:03.004057 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:03.004065 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:03.004088 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:03.004238 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:03.004247 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:03.005680 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:03.005683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:03.007131 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:03.007142 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:03.008562 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:03.008568 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:03.008655 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:03.008773 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 14:29:03.008888 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 14:29:03.009450 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":528,"spinlock_wait_cycles":5888,"thread_start_us":95,"threads_started":1}
I20260808 14:29:03.009644 12438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:03.003309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54580 (local address 127.7.125.66:44007)
0808 14:29:03.004228 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:03.004240 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:03.004442 (+ 202us) server_negotiation.cc:407] Connection header received
0808 14:29:03.004549 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:03.004553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:03.004732 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:03.004893 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:03.005296 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:03.006255 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:03.007601 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:03.008355 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:03.008447 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:03.008567 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:03.009152 (+ 585us) server_negotiation.cc:299] Negotiation successful
0808 14:29:03.009494 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":821,"thread_start_us":141,"threads_started":1}
I20260808 14:29:03.009912 12439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:03.003687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:44007 (local address 127.0.0.1:54580)
0808 14:29:03.004333 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:03.004345 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:03.004444 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:03.005039 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:03.005041 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:03.005058 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:03.005160 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:03.005168 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:03.006401 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:03.006404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:03.007466 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:03.007477 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:03.008566 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:03.008585 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:03.008760 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:03.008779 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 14:29:03.008877 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 14:29:03.008969 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"mutex_wait_us":47,"thread_start_us":83,"threads_started":1}
I20260808 14:29:03.012504 12354 tablet_service.cc:1467] Processing CreateTablet for tablet d6cc37c121d04de68277a460e9ab0107 (DEFAULT_TABLE table=test-table [id=fd0b478ca4ad42c98131c83884e0245b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:03.012504 12074 tablet_service.cc:1467] Processing CreateTablet for tablet d6cc37c121d04de68277a460e9ab0107 (DEFAULT_TABLE table=test-table [id=fd0b478ca4ad42c98131c83884e0245b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:03.012589 12217 tablet_service.cc:1467] Processing CreateTablet for tablet d6cc37c121d04de68277a460e9ab0107 (DEFAULT_TABLE table=test-table [id=fd0b478ca4ad42c98131c83884e0245b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:03.013679 12074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6cc37c121d04de68277a460e9ab0107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:03.013681 12354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6cc37c121d04de68277a460e9ab0107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:03.013681 12217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6cc37c121d04de68277a460e9ab0107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:03.020187 12442 tablet_bootstrap.cc:492] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d: Bootstrap starting.
I20260808 14:29:03.021283 12444 tablet_bootstrap.cc:492] T d6cc37c121d04de68277a460e9ab0107 P 25a4d431a271489ea414b4df52509cf6: Bootstrap starting.
I20260808 14:29:03.023268 12443 tablet_bootstrap.cc:492] T d6cc37c121d04de68277a460e9ab0107 P 72383143a2504ac1aa63010aa84fa972: Bootstrap starting.
I20260808 14:29:03.023423 12442 tablet_bootstrap.cc:654] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:03.023438 12444 tablet_bootstrap.cc:654] T d6cc37c121d04de68277a460e9ab0107 P 25a4d431a271489ea414b4df52509cf6: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:03.024534 12442 log.cc:824] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:03.024667 12444 log.cc:824] T d6cc37c121d04de68277a460e9ab0107 P 25a4d431a271489ea414b4df52509cf6: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:03.025373 12443 tablet_bootstrap.cc:654] T d6cc37c121d04de68277a460e9ab0107 P 72383143a2504ac1aa63010aa84fa972: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:03.026139 12443 log.cc:824] T d6cc37c121d04de68277a460e9ab0107 P 72383143a2504ac1aa63010aa84fa972: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:03.026757 12442 tablet_bootstrap.cc:492] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d: No bootstrap required, opened a new log
I20260808 14:29:03.026959 12442 ts_tablet_manager.cc:1397] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 14:29:03.027890 12444 tablet_bootstrap.cc:492] T d6cc37c121d04de68277a460e9ab0107 P 25a4d431a271489ea414b4df52509cf6: No bootstrap required, opened a new log
I20260808 14:29:03.028093 12444 ts_tablet_manager.cc:1397] T d6cc37c121d04de68277a460e9ab0107 P 25a4d431a271489ea414b4df52509cf6: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 14:29:03.028242 12443 tablet_bootstrap.cc:492] T d6cc37c121d04de68277a460e9ab0107 P 72383143a2504ac1aa63010aa84fa972: No bootstrap required, opened a new log
I20260808 14:29:03.028421 12443 ts_tablet_manager.cc:1397] T d6cc37c121d04de68277a460e9ab0107 P 72383143a2504ac1aa63010aa84fa972: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 14:29:03.030958 12444 raft_consensus.cc:348] T d6cc37c121d04de68277a460e9ab0107 P 25a4d431a271489ea414b4df52509cf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72383143a2504ac1aa63010aa84fa972" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37795 } } peers { permanent_uuid: "25a4d431a271489ea414b4df52509cf6" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40191 } } peers { permanent_uuid: "8e0fec4a16a34ccab687363caadd063d" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44007 } }
I20260808 14:29:03.031023 12442 raft_consensus.cc:348] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72383143a2504ac1aa63010aa84fa972" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37795 } } peers { permanent_uuid: "25a4d431a271489ea414b4df52509cf6" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40191 } } peers { permanent_uuid: "8e0fec4a16a34ccab687363caadd063d" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44007 } }
I20260808 14:29:03.031301 12444 raft_consensus.cc:374] T d6cc37c121d04de68277a460e9ab0107 P 25a4d431a271489ea414b4df52509cf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:03.031301 12442 raft_consensus.cc:374] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:03.031358 12444 raft_consensus.cc:729] T d6cc37c121d04de68277a460e9ab0107 P 25a4d431a271489ea414b4df52509cf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25a4d431a271489ea414b4df52509cf6, State: Initialized, Role: FOLLOWER
I20260808 14:29:03.031365 12442 raft_consensus.cc:729] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e0fec4a16a34ccab687363caadd063d, State: Initialized, Role: FOLLOWER
I20260808 14:29:03.031810 12444 consensus_queue.cc:260] T d6cc37c121d04de68277a460e9ab0107 P 25a4d431a271489ea414b4df52509cf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72383143a2504ac1aa63010aa84fa972" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37795 } } peers { permanent_uuid: "25a4d431a271489ea414b4df52509cf6" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40191 } } peers { permanent_uuid: "8e0fec4a16a34ccab687363caadd063d" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44007 } }
I20260808 14:29:03.031816 12442 consensus_queue.cc:260] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72383143a2504ac1aa63010aa84fa972" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37795 } } peers { permanent_uuid: "25a4d431a271489ea414b4df52509cf6" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40191 } } peers { permanent_uuid: "8e0fec4a16a34ccab687363caadd063d" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44007 } }
I20260808 14:29:03.032516 12424 heartbeater.cc:503] Master 127.7.125.126:33635 was elected leader, sending a full tablet report...
I20260808 14:29:03.032832 12444 ts_tablet_manager.cc:1428] T d6cc37c121d04de68277a460e9ab0107 P 25a4d431a271489ea414b4df52509cf6: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260808 14:29:03.032836 12442 ts_tablet_manager.cc:1428] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260808 14:29:03.033041 12284 heartbeater.cc:503] Master 127.7.125.126:33635 was elected leader, sending a full tablet report...
I20260808 14:29:03.033434 12443 raft_consensus.cc:348] T d6cc37c121d04de68277a460e9ab0107 P 72383143a2504ac1aa63010aa84fa972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72383143a2504ac1aa63010aa84fa972" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37795 } } peers { permanent_uuid: "25a4d431a271489ea414b4df52509cf6" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40191 } } peers { permanent_uuid: "8e0fec4a16a34ccab687363caadd063d" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44007 } }
I20260808 14:29:03.033646 12443 raft_consensus.cc:374] T d6cc37c121d04de68277a460e9ab0107 P 72383143a2504ac1aa63010aa84fa972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:03.033706 12443 raft_consensus.cc:729] T d6cc37c121d04de68277a460e9ab0107 P 72383143a2504ac1aa63010aa84fa972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72383143a2504ac1aa63010aa84fa972, State: Initialized, Role: FOLLOWER
I20260808 14:29:03.034196 12443 consensus_queue.cc:260] T d6cc37c121d04de68277a460e9ab0107 P 72383143a2504ac1aa63010aa84fa972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72383143a2504ac1aa63010aa84fa972" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37795 } } peers { permanent_uuid: "25a4d431a271489ea414b4df52509cf6" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40191 } } peers { permanent_uuid: "8e0fec4a16a34ccab687363caadd063d" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44007 } }
I20260808 14:29:03.035136 12443 ts_tablet_manager.cc:1428] T d6cc37c121d04de68277a460e9ab0107 P 72383143a2504ac1aa63010aa84fa972: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 14:29:03.035382 12144 heartbeater.cc:503] Master 127.7.125.126:33635 was elected leader, sending a full tablet report...
W20260808 14:29:03.199806 12425 tablet.cc:2369] T d6cc37c121d04de68277a460e9ab0107 P 25a4d431a271489ea414b4df52509cf6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:29:03.213267 12285 tablet.cc:2369] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:29:03.235733 12448 raft_consensus.cc:482] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:29:03.235961 12448 raft_consensus.cc:504] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72383143a2504ac1aa63010aa84fa972" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37795 } } peers { permanent_uuid: "25a4d431a271489ea414b4df52509cf6" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40191 } } peers { permanent_uuid: "8e0fec4a16a34ccab687363caadd063d" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44007 } }
I20260808 14:29:03.237252 12448 leader_election.cc:288] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72383143a2504ac1aa63010aa84fa972 (127.7.125.65:37795), 25a4d431a271489ea414b4df52509cf6 (127.7.125.67:40191)
W20260808 14:29:03.239329 12145 tablet.cc:2369] T d6cc37c121d04de68277a460e9ab0107 P 72383143a2504ac1aa63010aa84fa972: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:29:03.242051 12451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:03.237479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:37795 (local address 127.7.125.66:50215)
0808 14:29:03.237820 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:03.237837 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:03.237999 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:03.238455 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:03.238458 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:03.238477 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:03.238618 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:03.238624 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:03.239542 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:03.239545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:03.240947 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:03.240958 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:03.241599 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:03.241605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:03.241706 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:03.241735 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 14:29:03.241805 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 14:29:03.241892 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":34,"thread_start_us":136,"threads_started":1}
I20260808 14:29:03.242063 12437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:03.237683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:50215 (local address 127.7.125.65:37795)
0808 14:29:03.237821 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:03.237826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:03.237910 (+ 84us) server_negotiation.cc:407] Connection header received
0808 14:29:03.238138 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:03.238144 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:03.238195 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:03.238247 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:03.238750 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:03.239388 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:03.241081 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:03.241475 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:03.241571 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:03.241649 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:03.241843 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 14:29:03.241940 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 14:29:03.242125 12441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:03.238085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:36861 (local address 127.7.125.67:40191)
0808 14:29:03.238213 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:03.238219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:03.238327 (+ 108us) server_negotiation.cc:407] Connection header received
0808 14:29:03.238465 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:03.238468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:03.238514 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:03.238598 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:03.238908 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:03.239602 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:03.241116 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:03.241505 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:03.241644 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:03.241699 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:03.241928 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 14:29:03.242003 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 14:29:03.242092 12452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:03.237910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:40191 (local address 127.7.125.66:36861)
0808 14:29:03.238231 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:03.238247 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:03.238353 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:03.238618 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:03.238620 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:03.238636 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:03.238784 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:03.238791 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:03.239712 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:03.239716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:03.240957 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:03.240964 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:03.241658 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:03.241664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:03.241771 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:03.241810 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 14:29:03.241892 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 14:29:03.241960 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":51,"thread_start_us":71,"threads_started":1}
I20260808 14:29:03.242926 12096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6cc37c121d04de68277a460e9ab0107" candidate_uuid: "8e0fec4a16a34ccab687363caadd063d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72383143a2504ac1aa63010aa84fa972" is_pre_election: true
I20260808 14:29:03.242926 12376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6cc37c121d04de68277a460e9ab0107" candidate_uuid: "8e0fec4a16a34ccab687363caadd063d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25a4d431a271489ea414b4df52509cf6" is_pre_election: true
I20260808 14:29:03.243261 12096 raft_consensus.cc:2454] T d6cc37c121d04de68277a460e9ab0107 P 72383143a2504ac1aa63010aa84fa972 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e0fec4a16a34ccab687363caadd063d in term 0.
I20260808 14:29:03.243261 12376 raft_consensus.cc:2454] T d6cc37c121d04de68277a460e9ab0107 P 25a4d431a271489ea414b4df52509cf6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e0fec4a16a34ccab687363caadd063d in term 0.
I20260808 14:29:03.243767 12167 leader_election.cc:302] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72383143a2504ac1aa63010aa84fa972, 8e0fec4a16a34ccab687363caadd063d; no voters:
I20260808 14:29:03.244057 12448 raft_consensus.cc:2781] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:29:03.244136 12448 raft_consensus.cc:482] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:29:03.244166 12448 raft_consensus.cc:3037] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:03.245234 12448 raft_consensus.cc:504] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72383143a2504ac1aa63010aa84fa972" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37795 } } peers { permanent_uuid: "25a4d431a271489ea414b4df52509cf6" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40191 } } peers { permanent_uuid: "8e0fec4a16a34ccab687363caadd063d" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44007 } }
I20260808 14:29:03.245586 12448 leader_election.cc:288] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d [CANDIDATE]: Term 1 election: Requested vote from peers 72383143a2504ac1aa63010aa84fa972 (127.7.125.65:37795), 25a4d431a271489ea414b4df52509cf6 (127.7.125.67:40191)
I20260808 14:29:03.245976 12096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6cc37c121d04de68277a460e9ab0107" candidate_uuid: "8e0fec4a16a34ccab687363caadd063d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72383143a2504ac1aa63010aa84fa972"
I20260808 14:29:03.246096 12376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6cc37c121d04de68277a460e9ab0107" candidate_uuid: "8e0fec4a16a34ccab687363caadd063d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25a4d431a271489ea414b4df52509cf6"
I20260808 14:29:03.246134 12096 raft_consensus.cc:3037] T d6cc37c121d04de68277a460e9ab0107 P 72383143a2504ac1aa63010aa84fa972 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:03.246253 12376 raft_consensus.cc:3037] T d6cc37c121d04de68277a460e9ab0107 P 25a4d431a271489ea414b4df52509cf6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:03.247355 12096 raft_consensus.cc:2454] T d6cc37c121d04de68277a460e9ab0107 P 72383143a2504ac1aa63010aa84fa972 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e0fec4a16a34ccab687363caadd063d in term 1.
I20260808 14:29:03.247457 12376 raft_consensus.cc:2454] T d6cc37c121d04de68277a460e9ab0107 P 25a4d431a271489ea414b4df52509cf6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e0fec4a16a34ccab687363caadd063d in term 1.
I20260808 14:29:03.247737 12167 leader_election.cc:302] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72383143a2504ac1aa63010aa84fa972, 8e0fec4a16a34ccab687363caadd063d; no voters:
I20260808 14:29:03.247918 12448 raft_consensus.cc:2781] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:29:03.248214 12448 raft_consensus.cc:686] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d [term 1 LEADER]: Becoming Leader. State: Replica: 8e0fec4a16a34ccab687363caadd063d, State: Running, Role: LEADER
I20260808 14:29:03.248554 12448 consensus_queue.cc:237] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72383143a2504ac1aa63010aa84fa972" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37795 } } peers { permanent_uuid: "25a4d431a271489ea414b4df52509cf6" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40191 } } peers { permanent_uuid: "8e0fec4a16a34ccab687363caadd063d" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44007 } }
I20260808 14:29:03.251984 11944 catalog_manager.cc:5495] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e0fec4a16a34ccab687363caadd063d (127.7.125.66). New cstate: current_term: 1 leader_uuid: "8e0fec4a16a34ccab687363caadd063d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72383143a2504ac1aa63010aa84fa972" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25a4d431a271489ea414b4df52509cf6" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e0fec4a16a34ccab687363caadd063d" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44007 } health_report { overall_health: HEALTHY } } }
I20260808 14:29:03.303602 12438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:03.299890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54588 (local address 127.7.125.66:44007)
0808 14:29:03.300059 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:03.300064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:03.300080 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:29:03.300179 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:03.300182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:03.300249 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:03.300368 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:29:03.300632 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:03.301150 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:03.302021 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:03.302253 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:03.302360 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:03.302708 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:29:03.303205 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:29:03.303347 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:03.303385 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 14:29:03.303455 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 14:29:03.309617 12376 raft_consensus.cc:1261] T d6cc37c121d04de68277a460e9ab0107 P 25a4d431a271489ea414b4df52509cf6 [term 1 FOLLOWER]: Refusing update from remote peer 8e0fec4a16a34ccab687363caadd063d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:29:03.309661 12096 raft_consensus.cc:1261] T d6cc37c121d04de68277a460e9ab0107 P 72383143a2504ac1aa63010aa84fa972 [term 1 FOLLOWER]: Refusing update from remote peer 8e0fec4a16a34ccab687363caadd063d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:29:03.310341 12448 consensus_queue.cc:1035] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d [LEADER]: Connected to new peer: Peer: permanent_uuid: "25a4d431a271489ea414b4df52509cf6" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 40191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:29:03.310529 12453 consensus_queue.cc:1035] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d [LEADER]: Connected to new peer: Peer: permanent_uuid: "72383143a2504ac1aa63010aa84fa972" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:29:03.319293 12458 mvcc.cc:205] Tried to move back new op lower bound from 7316272510188855296 to 7316272509948743680. Current Snapshot: MvccSnapshot[applied={T|T < 7316272510188855296}]
I20260808 14:29:03.320560 12459 mvcc.cc:205] Tried to move back new op lower bound from 7316272510188855296 to 7316272509948743680. Current Snapshot: MvccSnapshot[applied={T|T < 7316272510188855296}]
I20260808 14:29:03.326772 11944 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:29:03.326960 11944 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:29:03.329551 11944 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5abf19171918479b85311c66044e83cb: Sending DeleteTablet for 3 replicas of tablet d6cc37c121d04de68277a460e9ab0107
I20260808 14:29:03.330525 12074 tablet_service.cc:1514] Processing DeleteTablet for tablet d6cc37c121d04de68277a460e9ab0107 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:03 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:41554
I20260808 14:29:03.330760 12354 tablet_service.cc:1514] Processing DeleteTablet for tablet d6cc37c121d04de68277a460e9ab0107 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:03 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:51692
I20260808 14:29:03.331261 12217 tablet_service.cc:1514] Processing DeleteTablet for tablet d6cc37c121d04de68277a460e9ab0107 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:03 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:54580
I20260808 14:29:03.331506 12467 tablet_replica.cc:330] T d6cc37c121d04de68277a460e9ab0107 P 72383143a2504ac1aa63010aa84fa972: stopping tablet replica
I20260808 14:29:03.331739 12469 tablet_replica.cc:330] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d: stopping tablet replica
I20260808 14:29:03.331817 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 12007
I20260808 14:29:03.332888 12469 raft_consensus.cc:2229] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:29:03.333338 12469 raft_consensus.cc:2258] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:03.335078 12469 ts_tablet_manager.cc:1905] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:29:03.337433 12468 tablet_replica.cc:330] T d6cc37c121d04de68277a460e9ab0107 P 25a4d431a271489ea414b4df52509cf6: stopping tablet replica
I20260808 14:29:03.337906 12469 ts_tablet_manager.cc:1918] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:29:03.337894 12468 raft_consensus.cc:2229] T d6cc37c121d04de68277a460e9ab0107 P 25a4d431a271489ea414b4df52509cf6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:29:03.338013 12469 log.cc:1196] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/d6cc37c121d04de68277a460e9ab0107
I20260808 14:29:03.338335 12469 ts_tablet_manager.cc:1939] T d6cc37c121d04de68277a460e9ab0107 P 8e0fec4a16a34ccab687363caadd063d: Deleting consensus metadata
I20260808 14:29:03.338467 12468 raft_consensus.cc:2258] T d6cc37c121d04de68277a460e9ab0107 P 25a4d431a271489ea414b4df52509cf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:03.339636 11930 catalog_manager.cc:4841] TS 8e0fec4a16a34ccab687363caadd063d (127.7.125.66:44007): tablet d6cc37c121d04de68277a460e9ab0107 (table test-table [id=fd0b478ca4ad42c98131c83884e0245b]) successfully deleted
I20260808 14:29:03.340384 12468 ts_tablet_manager.cc:1905] T d6cc37c121d04de68277a460e9ab0107 P 25a4d431a271489ea414b4df52509cf6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:29:03.343875 12468 ts_tablet_manager.cc:1918] T d6cc37c121d04de68277a460e9ab0107 P 25a4d431a271489ea414b4df52509cf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:29:03.343993 12468 log.cc:1196] T d6cc37c121d04de68277a460e9ab0107 P 25a4d431a271489ea414b4df52509cf6: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/d6cc37c121d04de68277a460e9ab0107
I20260808 14:29:03.344357 12468 ts_tablet_manager.cc:1939] T d6cc37c121d04de68277a460e9ab0107 P 25a4d431a271489ea414b4df52509cf6: Deleting consensus metadata
W20260808 14:29:03.344774 11929 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.125.65:37795 (error 108)
I20260808 14:29:03.345702 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 12149
I20260808 14:29:03.345711 11929 catalog_manager.cc:4841] TS 25a4d431a271489ea414b4df52509cf6 (127.7.125.67:40191): tablet d6cc37c121d04de68277a460e9ab0107 (table test-table [id=fd0b478ca4ad42c98131c83884e0245b]) successfully deleted
I20260808 14:29:03.347651 12436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:03.345563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:37795 (local address 127.0.0.1:41564)
0808 14:29:03.347485 (+ 1922us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:03.347560 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.65:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1827}
W20260808 14:29:03.347882 11929 catalog_manager.cc:4584] TS 72383143a2504ac1aa63010aa84fa972 (127.7.125.65:37795): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d6cc37c121d04de68277a460e9ab0107: Network error: Client connection negotiation failed: client connection to 127.7.125.65:37795: connect: Connection refused (error 111)
I20260808 14:29:03.353888 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 12290
I20260808 14:29:03.360992 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 11911
2026-08-08T14:29:03Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2694 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T14:29:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:29:03Z Disabled control of system clock
I20260808 14:29:03.395824 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:40453
--webserver_interface=127.7.125.126
--webserver_port=0
--builtin_ntp_servers=127.7.125.84:43011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:40453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 14:29:03.509006 12474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:03.509303 12474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:03.509352 12474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:03.513518 12474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:29:03.513592 12474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:03.513607 12474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:03.513624 12474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:29:03.513640 12474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:29:03.518805 12474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:43011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:40453
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:40453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:03.520030 12474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:03.521493 12474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:03.530174 12483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:03.530287 12474 server_base.cc:1034] running on GCE node
W20260808 14:29:03.530174 12481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:03.530192 12480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:03.531100 12474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:03.532223 12474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:03.533386 12474 hybrid_clock.cc:648] HybridClock initialized: now 1786199343533369 us; error 32 us; skew 500 ppm
I20260808 14:29:03.535422 12474 webserver.cc:460] Webserver started at http://127.7.125.126:33425/ using document root <none> and password file <none>
I20260808 14:29:03.536024 12474 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:03.536088 12474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:03.536351 12474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:03.538614 12474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "2bdbb47ea69e4863bfdaa025fd4c84d3"
format_stamp: "Formatted at 2026-08-08 14:29:03 on dist-test-slave-w206"
server_key: "39887cfa2686d9c12e732dc3bed37f8c"
server_key_iv: "2c1f9393f7067250e0cb888c10172827"
server_key_version: "encryptionkey@0"
I20260808 14:29:03.539234 12474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "2bdbb47ea69e4863bfdaa025fd4c84d3"
format_stamp: "Formatted at 2026-08-08 14:29:03 on dist-test-slave-w206"
server_key: "39887cfa2686d9c12e732dc3bed37f8c"
server_key_iv: "2c1f9393f7067250e0cb888c10172827"
server_key_version: "encryptionkey@0"
I20260808 14:29:03.543223 12474 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:29:03.545698 12489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:03.547098 12474 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:29:03.547240 12474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "2bdbb47ea69e4863bfdaa025fd4c84d3"
format_stamp: "Formatted at 2026-08-08 14:29:03 on dist-test-slave-w206"
server_key: "39887cfa2686d9c12e732dc3bed37f8c"
server_key_iv: "2c1f9393f7067250e0cb888c10172827"
server_key_version: "encryptionkey@0"
I20260808 14:29:03.547364 12474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:03.552150 12474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:03.552795 12474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:03.552999 12474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:03.559953 12474 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:40453
I20260808 14:29:03.559971 12545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:40453 every 8 connection(s)
I20260808 14:29:03.560917 12474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:29:03.562584 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 12474
I20260808 14:29:03.562706 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:29:03.563060 7669 external_mini_cluster.cc:1428] Setting key 13a256d00cacf3eb045907e994f955a6
I20260808 14:29:03.564836 12546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:03.568819 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:03.564626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43520 (local address 127.7.125.126:40453)
0808 14:29:03.565331 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:03.565346 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:03.565381 (+ 35us) server_negotiation.cc:407] Connection header received
0808 14:29:03.566038 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:03.566053 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:03.566385 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:03.566739 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:03.567562 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:03.567567 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:03.567599 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:29:03.567622 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:03.567673 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:29:03.567689 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:03.567845 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:03.567917 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 14:29:03.568156 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":145,"threads_started":1}
I20260808 14:29:03.570595 12546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3: Bootstrap starting.
I20260808 14:29:03.572968 12546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:03.573701 12546 log.cc:824] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:03.575652 12546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3: No bootstrap required, opened a new log
I20260808 14:29:03.579399 12546 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bdbb47ea69e4863bfdaa025fd4c84d3" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 40453 } }
I20260808 14:29:03.579633 12546 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:03.579710 12546 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bdbb47ea69e4863bfdaa025fd4c84d3, State: Initialized, Role: FOLLOWER
I20260808 14:29:03.580190 12546 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bdbb47ea69e4863bfdaa025fd4c84d3" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 40453 } }
I20260808 14:29:03.580343 12546 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:29:03.580413 12546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:29:03.580541 12546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:03.581583 12546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bdbb47ea69e4863bfdaa025fd4c84d3" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 40453 } }
I20260808 14:29:03.581923 12546 leader_election.cc:302] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2bdbb47ea69e4863bfdaa025fd4c84d3; no voters:
I20260808 14:29:03.582247 12546 leader_election.cc:288] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:29:03.582382 12551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:29:03.582643 12551 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3 [term 1 LEADER]: Becoming Leader. State: Replica: 2bdbb47ea69e4863bfdaa025fd4c84d3, State: Running, Role: LEADER
I20260808 14:29:03.582957 12551 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bdbb47ea69e4863bfdaa025fd4c84d3" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 40453 } }
I20260808 14:29:03.583508 12546 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:29:03.584880 12552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2bdbb47ea69e4863bfdaa025fd4c84d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bdbb47ea69e4863bfdaa025fd4c84d3" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 40453 } } }
I20260808 14:29:03.584888 12553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bdbb47ea69e4863bfdaa025fd4c84d3. Latest consensus state: current_term: 1 leader_uuid: "2bdbb47ea69e4863bfdaa025fd4c84d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bdbb47ea69e4863bfdaa025fd4c84d3" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 40453 } } }
I20260808 14:29:03.585058 12552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3 [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:03.585079 12553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3 [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:03.585441 12560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:29:03.588613 12560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:29:03.594033 12560 catalog_manager.cc:1347] Generated new cluster ID: 99bdf2f7936047e38d1909dc844a70e8
I20260808 14:29:03.594111 12560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:29:03.606066 12560 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:29:03.606546 12560 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:29:03.613983 12560 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3: Generated new TSK 0
I20260808 14:29:03.614888 12560 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:29:03.625756 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:40453
--builtin_ntp_servers=127.7.125.84:43011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 14:29:03.755759 12570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:03.756058 12570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:03.756145 12570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:03.760820 12570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:03.760895 12570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:03.761003 12570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:29:03.766456 12570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:43011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:40453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:03.767642 12570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:03.769176 12570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:03.779639 12576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:03.779745 12577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:03.779639 12581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:03.779855 12570 server_base.cc:1034] running on GCE node
I20260808 14:29:03.780434 12570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:03.781098 12570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:03.782289 12570 hybrid_clock.cc:648] HybridClock initialized: now 1786199343782275 us; error 27 us; skew 500 ppm
I20260808 14:29:03.784263 12570 webserver.cc:460] Webserver started at http://127.7.125.65:40721/ using document root <none> and password file <none>
I20260808 14:29:03.784852 12570 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:03.784950 12570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:03.785171 12570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:03.787398 12570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "431c460ae7dd4d32962a5f90ffffc0a6"
format_stamp: "Formatted at 2026-08-08 14:29:03 on dist-test-slave-w206"
server_key: "afef29c78b379377a1e3faabce06a0ea"
server_key_iv: "e2ab44a7b82663d73d1c056c2b629eee"
server_key_version: "encryptionkey@0"
I20260808 14:29:03.787953 12570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "431c460ae7dd4d32962a5f90ffffc0a6"
format_stamp: "Formatted at 2026-08-08 14:29:03 on dist-test-slave-w206"
server_key: "afef29c78b379377a1e3faabce06a0ea"
server_key_iv: "e2ab44a7b82663d73d1c056c2b629eee"
server_key_version: "encryptionkey@0"
I20260808 14:29:03.791785 12570 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 14:29:03.794123 12589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:03.795451 12570 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:29:03.795594 12570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "431c460ae7dd4d32962a5f90ffffc0a6"
format_stamp: "Formatted at 2026-08-08 14:29:03 on dist-test-slave-w206"
server_key: "afef29c78b379377a1e3faabce06a0ea"
server_key_iv: "e2ab44a7b82663d73d1c056c2b629eee"
server_key_version: "encryptionkey@0"
I20260808 14:29:03.795709 12570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:03.800251 12570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:03.800855 12570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:03.801054 12570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:03.801656 12570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:03.802559 12570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:03.802630 12570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:03.802698 12570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:03.802732 12570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:03.811446 12570 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:36761
I20260808 14:29:03.811470 12708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:36761 every 8 connection(s)
I20260808 14:29:03.812350 12570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:29:03.816258 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 12570
I20260808 14:29:03.816444 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:29:03.816805 7669 external_mini_cluster.cc:1428] Setting key 85c503eda11db95d8bc9d081e42c8ac0
I20260808 14:29:03.819090 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:40453
--builtin_ntp_servers=127.7.125.84:43011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 14:29:03.819880 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:03.814242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:44615 (local address 127.7.125.126:40453)
0808 14:29:03.814404 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:03.814408 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:03.815492 (+ 1084us) server_negotiation.cc:407] Connection header received
0808 14:29:03.817301 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:03.817304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:03.817352 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:03.817474 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:03.819453 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:03.819456 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:03.819458 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:29:03.819469 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:03.819489 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:29:03.819497 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:03.819571 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:03.819742 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 14:29:03.819778 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:29:03.821071 12712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:03.814637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:40453 (local address 127.7.125.65:44615)
0808 14:29:03.815345 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:03.815386 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:03.817040 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:03.817666 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:03.817673 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:03.817761 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:03.819132 (+ 1371us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:29:03.819134 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:03.819192 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:29:03.819289 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:03.819586 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:03.819598 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:03.819607 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 14:29:03.819787 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 14:29:03.820049 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":118,"threads_started":1}
I20260808 14:29:03.822397 12709 heartbeater.cc:347] Connected to a master server at 127.7.125.126:40453
I20260808 14:29:03.822633 12709 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:03.823102 12709 heartbeater.cc:511] Master 127.7.125.126:40453 requested a full tablet report, sending...
I20260808 14:29:03.824955 12506 ts_manager.cc:194] Registered new tserver with Master: 431c460ae7dd4d32962a5f90ffffc0a6 (127.7.125.65:36761)
W20260808 14:29:03.936722 12713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:03.937011 12713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:03.937052 12713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:03.941094 12713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:03.941156 12713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:03.941215 12713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:29:03.946611 12713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:43011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:40453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:03.947738 12713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:03.949160 12713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:03.959162 12719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:03.959239 12713 server_base.cc:1034] running on GCE node
W20260808 14:29:03.959208 12718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:03.959170 12721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:03.959827 12713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:03.960469 12713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:03.961632 12713 hybrid_clock.cc:648] HybridClock initialized: now 1786199343961620 us; error 28 us; skew 500 ppm
I20260808 14:29:03.963615 12713 webserver.cc:460] Webserver started at http://127.7.125.66:46625/ using document root <none> and password file <none>
I20260808 14:29:03.964208 12713 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:03.964272 12713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:03.964519 12713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:03.966758 12713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "b249c9a397ef47d9b5a9084032aea595"
format_stamp: "Formatted at 2026-08-08 14:29:03 on dist-test-slave-w206"
server_key: "30058d1732dbb58f917318d3b01163b6"
server_key_iv: "50c02773e9e53d1f85f50874d34b96c8"
server_key_version: "encryptionkey@0"
I20260808 14:29:03.967422 12713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "b249c9a397ef47d9b5a9084032aea595"
format_stamp: "Formatted at 2026-08-08 14:29:03 on dist-test-slave-w206"
server_key: "30058d1732dbb58f917318d3b01163b6"
server_key_iv: "50c02773e9e53d1f85f50874d34b96c8"
server_key_version: "encryptionkey@0"
I20260808 14:29:03.971328 12713 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:29:03.973634 12727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:03.974843 12713 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:29:03.975015 12713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "b249c9a397ef47d9b5a9084032aea595"
format_stamp: "Formatted at 2026-08-08 14:29:03 on dist-test-slave-w206"
server_key: "30058d1732dbb58f917318d3b01163b6"
server_key_iv: "50c02773e9e53d1f85f50874d34b96c8"
server_key_version: "encryptionkey@0"
I20260808 14:29:03.975169 12713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:03.979846 12713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:03.980418 12713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:03.980556 12713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:03.981051 12713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:03.981925 12713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:03.981971 12713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:03.982008 12713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:03.982023 12713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:03.991289 12713 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:38159
I20260808 14:29:03.991320 12847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:38159 every 8 connection(s)
I20260808 14:29:03.992202 12713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:29:03.997161 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 12713
I20260808 14:29:03.997368 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:29:03.997720 7669 external_mini_cluster.cc:1428] Setting key 1a2fa73d18f19fa5bb5932f99a3b499c
I20260808 14:29:03.998652 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:03.994036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:46677 (local address 127.7.125.126:40453)
0808 14:29:03.994190 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:03.994193 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:03.995325 (+ 1132us) server_negotiation.cc:407] Connection header received
0808 14:29:03.996669 (+ 1344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:03.996672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:03.996728 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:03.996835 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:03.998062 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:03.998066 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:03.998068 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:29:03.998084 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:03.998110 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:29:03.998120 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:03.998216 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:03.998481 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 14:29:03.998526 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 14:29:03.999804 12850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:03.994445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:40453 (local address 127.7.125.66:46677)
0808 14:29:03.995174 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:03.995216 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:03.996447 (+ 1231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:03.997010 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:03.997017 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:03.997106 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:03.997751 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:29:03.997752 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:03.997826 (+ 74us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:29:03.997912 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:03.998260 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:03.998274 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:03.998284 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 14:29:03.998573 (+ 289us) client_negotiation.cc:240] Negotiation successful
0808 14:29:03.998813 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":114,"threads_started":1}
I20260808 14:29:04.000145 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:40453
--builtin_ntp_servers=127.7.125.84:43011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 14:29:04.001065 12848 heartbeater.cc:347] Connected to a master server at 127.7.125.126:40453
I20260808 14:29:04.001314 12848 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:04.001791 12848 heartbeater.cc:511] Master 127.7.125.126:40453 requested a full tablet report, sending...
I20260808 14:29:04.003093 12506 ts_manager.cc:194] Registered new tserver with Master: b249c9a397ef47d9b5a9084032aea595 (127.7.125.66:38159)
W20260808 14:29:04.117012 12851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:04.117273 12851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:04.117322 12851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:04.121846 12851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:04.121907 12851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:04.121963 12851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:29:04.126753 12851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:43011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:40453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:04.128015 12851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:04.129354 12851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:04.139005 12859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:04.139050 12851 server_base.cc:1034] running on GCE node
W20260808 14:29:04.139001 12856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:04.139001 12857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:04.139678 12851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:04.140342 12851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:04.141535 12851 hybrid_clock.cc:648] HybridClock initialized: now 1786199344141521 us; error 25 us; skew 500 ppm
I20260808 14:29:04.143440 12851 webserver.cc:460] Webserver started at http://127.7.125.67:37209/ using document root <none> and password file <none>
I20260808 14:29:04.144090 12851 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:04.144191 12851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:04.144426 12851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:04.146548 12851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "a5d08b3e408542239645ed925ebcb9eb"
format_stamp: "Formatted at 2026-08-08 14:29:04 on dist-test-slave-w206"
server_key: "81a9cc7372ab343b565cd9e804ab4d4c"
server_key_iv: "c57a4693ae3fada5e2dc4f27711ea7ab"
server_key_version: "encryptionkey@0"
I20260808 14:29:04.147251 12851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "a5d08b3e408542239645ed925ebcb9eb"
format_stamp: "Formatted at 2026-08-08 14:29:04 on dist-test-slave-w206"
server_key: "81a9cc7372ab343b565cd9e804ab4d4c"
server_key_iv: "c57a4693ae3fada5e2dc4f27711ea7ab"
server_key_version: "encryptionkey@0"
I20260808 14:29:04.151454 12851 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:29:04.154093 12865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:04.155472 12851 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:29:04.155612 12851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "a5d08b3e408542239645ed925ebcb9eb"
format_stamp: "Formatted at 2026-08-08 14:29:04 on dist-test-slave-w206"
server_key: "81a9cc7372ab343b565cd9e804ab4d4c"
server_key_iv: "c57a4693ae3fada5e2dc4f27711ea7ab"
server_key_version: "encryptionkey@0"
I20260808 14:29:04.155761 12851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:04.161207 12851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:04.161922 12851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:04.162119 12851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:04.162765 12851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:04.163918 12851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:04.163973 12851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:04.164016 12851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:04.164033 12851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:04.174046 12851 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:37161
I20260808 14:29:04.174067 12986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:37161 every 8 connection(s)
I20260808 14:29:04.175191 12851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:29:04.177984 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 12851
I20260808 14:29:04.178128 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:29:04.178432 7669 external_mini_cluster.cc:1428] Setting key ab83e65958811e117c76f3c22e816766
I20260808 14:29:04.182266 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:04.177190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:52027 (local address 127.7.125.126:40453)
0808 14:29:04.177340 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:04.177344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:04.178715 (+ 1371us) server_negotiation.cc:407] Connection header received
0808 14:29:04.180524 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:04.180528 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:04.180573 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:04.180645 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:04.181852 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:04.181855 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:04.181858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:29:04.181869 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:04.181890 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:29:04.181911 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:04.182000 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:04.182127 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 14:29:04.182164 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:29:04.183209 12990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:04.177670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:40453 (local address 127.7.125.67:52027)
0808 14:29:04.178521 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:04.178580 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:04.180307 (+ 1727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:04.180848 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:04.180856 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:04.180952 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:04.181590 (+ 638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:29:04.181592 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:04.181633 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:29:04.181702 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:04.182006 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:04.182012 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:04.182017 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 14:29:04.182146 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 14:29:04.182331 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":155,"threads_started":1}
I20260808 14:29:04.184273 12987 heartbeater.cc:347] Connected to a master server at 127.7.125.126:40453
I20260808 14:29:04.184527 12987 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:04.184995 12987 heartbeater.cc:511] Master 127.7.125.126:40453 requested a full tablet report, sending...
I20260808 14:29:04.185988 12506 ts_manager.cc:194] Registered new tserver with Master: a5d08b3e408542239645ed925ebcb9eb (127.7.125.67:37161)
I20260808 14:29:04.192641 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:29:04.196426 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:04.194636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43530 (local address 127.7.125.126:40453)
0808 14:29:04.194826 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:04.194831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:04.194985 (+ 154us) server_negotiation.cc:407] Connection header received
0808 14:29:04.195150 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:04.195152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:04.195186 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:04.195267 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:04.196036 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:04.196040 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:04.196042 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:29:04.196054 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:04.196074 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:29:04.196090 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:04.196178 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:04.196260 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 14:29:04.196301 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 14:29:04.199231 12506 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43530:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:29:04.202446 12506 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:29:04.213776 12999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:04.210537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:38159 (local address 127.0.0.1:49384)
0808 14:29:04.210913 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:04.210968 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:04.211093 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:04.212299 (+ 1206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:04.212305 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:04.212382 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:04.212979 (+ 597us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:29:04.212981 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:04.213004 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:29:04.213076 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:04.213409 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:04.213422 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:04.213432 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 14:29:04.213549 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 14:29:04.213630 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":1,"thread_start_us":97,"threads_started":1}
I20260808 14:29:04.213819 13001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:04.210629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49384 (local address 127.7.125.66:38159)
0808 14:29:04.211477 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:04.211496 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:04.211525 (+ 29us) server_negotiation.cc:407] Connection header received
0808 14:29:04.211581 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:04.211587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:04.211792 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:04.212009 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:04.213194 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:04.213202 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:04.213216 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:29:04.213231 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:04.213275 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:29:04.213302 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:04.213397 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:04.213575 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 14:29:04.213649 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":162,"threads_started":1}
I20260808 14:29:04.214071 13003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:04.211271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56054 (local address 127.7.125.67:37161)
0808 14:29:04.212183 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:04.212197 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:04.212221 (+ 24us) server_negotiation.cc:407] Connection header received
0808 14:29:04.212275 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:04.212280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:04.212441 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:04.212585 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:04.213272 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:04.213276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:04.213285 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:29:04.213293 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:04.213326 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:29:04.213345 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:04.213473 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:04.213848 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 14:29:04.213916 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":802,"thread_start_us":140,"threads_started":1}
I20260808 14:29:04.213788 13004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:04.210549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:37161 (local address 127.0.0.1:56054)
0808 14:29:04.211711 (+ 1162us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:04.211725 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:04.211803 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:04.212590 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:04.212593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:04.212609 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:04.213109 (+ 500us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:29:04.213112 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:04.213124 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:29:04.213154 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:04.213466 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:04.213469 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:04.213472 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 14:29:04.213552 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 14:29:04.213621 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1096,"thread_start_us":93,"threads_started":1}
I20260808 14:29:04.216238 13000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:04.211410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:36761 (local address 127.0.0.1:44126)
0808 14:29:04.213557 (+ 2147us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:04.213570 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:04.213643 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:04.214573 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:04.214576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:04.214591 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:04.215612 (+ 1021us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:29:04.215614 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:04.215624 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:29:04.215649 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:04.215993 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:04.215996 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:04.215999 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 14:29:04.216079 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 14:29:04.216116 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2075,"thread_start_us":82,"threads_started":1}
I20260808 14:29:04.216347 13005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:04.211582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44126 (local address 127.7.125.65:36761)
0808 14:29:04.212003 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:04.212015 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:04.213737 (+ 1722us) server_negotiation.cc:407] Connection header received
0808 14:29:04.213799 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:04.213804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:04.214423 (+ 619us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:04.214575 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:04.215784 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:04.215790 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:04.215801 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:29:04.215814 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:04.215861 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:29:04.215893 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:04.216020 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:04.216095 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 14:29:04.216159 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":133,"threads_started":1}
I20260808 14:29:04.217268 12777 tablet_service.cc:1467] Processing CreateTablet for tablet 0db305108a7b406bbff72738439082f7 (DEFAULT_TABLE table=test-table [id=8dc7cdd7a1ce45f18ee9d131031632e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:04.217644 12913 tablet_service.cc:1467] Processing CreateTablet for tablet 0db305108a7b406bbff72738439082f7 (DEFAULT_TABLE table=test-table [id=8dc7cdd7a1ce45f18ee9d131031632e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:04.218437 12777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0db305108a7b406bbff72738439082f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:04.218523 12913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0db305108a7b406bbff72738439082f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:04.218921 12638 tablet_service.cc:1467] Processing CreateTablet for tablet 0db305108a7b406bbff72738439082f7 (DEFAULT_TABLE table=test-table [id=8dc7cdd7a1ce45f18ee9d131031632e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:04.219820 12638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0db305108a7b406bbff72738439082f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:04.225562 13006 tablet_bootstrap.cc:492] T 0db305108a7b406bbff72738439082f7 P 431c460ae7dd4d32962a5f90ffffc0a6: Bootstrap starting.
I20260808 14:29:04.226533 13007 tablet_bootstrap.cc:492] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595: Bootstrap starting.
I20260808 14:29:04.226755 13008 tablet_bootstrap.cc:492] T 0db305108a7b406bbff72738439082f7 P a5d08b3e408542239645ed925ebcb9eb: Bootstrap starting.
I20260808 14:29:04.228562 13006 tablet_bootstrap.cc:654] T 0db305108a7b406bbff72738439082f7 P 431c460ae7dd4d32962a5f90ffffc0a6: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:04.228804 13008 tablet_bootstrap.cc:654] T 0db305108a7b406bbff72738439082f7 P a5d08b3e408542239645ed925ebcb9eb: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:04.228804 13007 tablet_bootstrap.cc:654] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:04.229377 13006 log.cc:824] T 0db305108a7b406bbff72738439082f7 P 431c460ae7dd4d32962a5f90ffffc0a6: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:04.229640 13008 log.cc:824] T 0db305108a7b406bbff72738439082f7 P a5d08b3e408542239645ed925ebcb9eb: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:04.229640 13007 log.cc:824] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:04.231606 13006 tablet_bootstrap.cc:492] T 0db305108a7b406bbff72738439082f7 P 431c460ae7dd4d32962a5f90ffffc0a6: No bootstrap required, opened a new log
I20260808 14:29:04.231606 13008 tablet_bootstrap.cc:492] T 0db305108a7b406bbff72738439082f7 P a5d08b3e408542239645ed925ebcb9eb: No bootstrap required, opened a new log
I20260808 14:29:04.231804 13006 ts_tablet_manager.cc:1397] T 0db305108a7b406bbff72738439082f7 P 431c460ae7dd4d32962a5f90ffffc0a6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:29:04.231804 13008 ts_tablet_manager.cc:1397] T 0db305108a7b406bbff72738439082f7 P a5d08b3e408542239645ed925ebcb9eb: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 14:29:04.231973 13007 tablet_bootstrap.cc:492] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595: No bootstrap required, opened a new log
I20260808 14:29:04.232211 13007 ts_tablet_manager.cc:1397] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 14:29:04.236632 13006 raft_consensus.cc:348] T 0db305108a7b406bbff72738439082f7 P 431c460ae7dd4d32962a5f90ffffc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b249c9a397ef47d9b5a9084032aea595" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38159 } } peers { permanent_uuid: "a5d08b3e408542239645ed925ebcb9eb" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37161 } } peers { permanent_uuid: "431c460ae7dd4d32962a5f90ffffc0a6" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 36761 } }
I20260808 14:29:04.236938 13006 raft_consensus.cc:374] T 0db305108a7b406bbff72738439082f7 P 431c460ae7dd4d32962a5f90ffffc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:04.237010 13006 raft_consensus.cc:729] T 0db305108a7b406bbff72738439082f7 P 431c460ae7dd4d32962a5f90ffffc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 431c460ae7dd4d32962a5f90ffffc0a6, State: Initialized, Role: FOLLOWER
I20260808 14:29:04.237255 13008 raft_consensus.cc:348] T 0db305108a7b406bbff72738439082f7 P a5d08b3e408542239645ed925ebcb9eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b249c9a397ef47d9b5a9084032aea595" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38159 } } peers { permanent_uuid: "a5d08b3e408542239645ed925ebcb9eb" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37161 } } peers { permanent_uuid: "431c460ae7dd4d32962a5f90ffffc0a6" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 36761 } }
I20260808 14:29:04.237509 13008 raft_consensus.cc:374] T 0db305108a7b406bbff72738439082f7 P a5d08b3e408542239645ed925ebcb9eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:04.237566 13008 raft_consensus.cc:729] T 0db305108a7b406bbff72738439082f7 P a5d08b3e408542239645ed925ebcb9eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5d08b3e408542239645ed925ebcb9eb, State: Initialized, Role: FOLLOWER
I20260808 14:29:04.237525 13006 consensus_queue.cc:260] T 0db305108a7b406bbff72738439082f7 P 431c460ae7dd4d32962a5f90ffffc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b249c9a397ef47d9b5a9084032aea595" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38159 } } peers { permanent_uuid: "a5d08b3e408542239645ed925ebcb9eb" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37161 } } peers { permanent_uuid: "431c460ae7dd4d32962a5f90ffffc0a6" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 36761 } }
I20260808 14:29:04.237743 13007 raft_consensus.cc:348] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b249c9a397ef47d9b5a9084032aea595" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38159 } } peers { permanent_uuid: "a5d08b3e408542239645ed925ebcb9eb" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37161 } } peers { permanent_uuid: "431c460ae7dd4d32962a5f90ffffc0a6" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 36761 } }
I20260808 14:29:04.238059 13007 raft_consensus.cc:374] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:04.238029 13008 consensus_queue.cc:260] T 0db305108a7b406bbff72738439082f7 P a5d08b3e408542239645ed925ebcb9eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b249c9a397ef47d9b5a9084032aea595" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38159 } } peers { permanent_uuid: "a5d08b3e408542239645ed925ebcb9eb" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37161 } } peers { permanent_uuid: "431c460ae7dd4d32962a5f90ffffc0a6" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 36761 } }
I20260808 14:29:04.238210 13007 raft_consensus.cc:729] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b249c9a397ef47d9b5a9084032aea595, State: Initialized, Role: FOLLOWER
I20260808 14:29:04.238691 13006 ts_tablet_manager.cc:1428] T 0db305108a7b406bbff72738439082f7 P 431c460ae7dd4d32962a5f90ffffc0a6: Time spent starting tablet: real 0.007s user 0.001s sys 0.006s
I20260808 14:29:04.238677 13007 consensus_queue.cc:260] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b249c9a397ef47d9b5a9084032aea595" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38159 } } peers { permanent_uuid: "a5d08b3e408542239645ed925ebcb9eb" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37161 } } peers { permanent_uuid: "431c460ae7dd4d32962a5f90ffffc0a6" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 36761 } }
I20260808 14:29:04.238934 13008 ts_tablet_manager.cc:1428] T 0db305108a7b406bbff72738439082f7 P a5d08b3e408542239645ed925ebcb9eb: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 14:29:04.239104 12709 heartbeater.cc:503] Master 127.7.125.126:40453 was elected leader, sending a full tablet report...
I20260808 14:29:04.239248 12987 heartbeater.cc:503] Master 127.7.125.126:40453 was elected leader, sending a full tablet report...
I20260808 14:29:04.240038 13007 ts_tablet_manager.cc:1428] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260808 14:29:04.240483 12848 heartbeater.cc:503] Master 127.7.125.126:40453 was elected leader, sending a full tablet report...
W20260808 14:29:04.243276 12849 tablet.cc:2369] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:29:04.313673 12711 tablet.cc:2369] T 0db305108a7b406bbff72738439082f7 P 431c460ae7dd4d32962a5f90ffffc0a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:29:04.359445 13014 raft_consensus.cc:482] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:29:04.359694 13014 raft_consensus.cc:504] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b249c9a397ef47d9b5a9084032aea595" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38159 } } peers { permanent_uuid: "a5d08b3e408542239645ed925ebcb9eb" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37161 } } peers { permanent_uuid: "431c460ae7dd4d32962a5f90ffffc0a6" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 36761 } }
I20260808 14:29:04.361114 13014 leader_election.cc:288] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5d08b3e408542239645ed925ebcb9eb (127.7.125.67:37161), 431c460ae7dd4d32962a5f90ffffc0a6 (127.7.125.65:36761)
I20260808 14:29:04.363332 13003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:04.361678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:56553 (local address 127.7.125.67:37161)
0808 14:29:04.361812 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:04.361817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:04.361959 (+ 142us) server_negotiation.cc:407] Connection header received
0808 14:29:04.362071 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:04.362074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:04.362117 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:04.362180 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:04.362810 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:04.362814 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:04.362817 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:29:04.362830 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:04.362852 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:29:04.362863 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:04.362964 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:04.363155 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 14:29:04.363199 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 14:29:04.363332 13005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:04.361516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:34047 (local address 127.7.125.65:36761)
0808 14:29:04.361648 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:04.361652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:04.361668 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:29:04.361834 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:04.361838 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:04.361896 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:04.361998 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:04.362810 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:04.362814 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:04.362817 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:29:04.362830 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:04.362852 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:29:04.362863 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:04.362997 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:04.363155 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 14:29:04.363199 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 14:29:04.363364 12850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:04.361413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:36761 (local address 127.7.125.66:34047)
0808 14:29:04.361553 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:04.361571 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:04.361675 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:04.362044 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:04.362048 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:04.362064 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:04.362557 (+ 493us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:29:04.362559 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:04.362607 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:29:04.362645 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:04.363025 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:04.363029 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:04.363034 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 14:29:04.363145 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 14:29:04.363191 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 14:29:04.363620 13015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:04.361419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:37161 (local address 127.7.125.66:56553)
0808 14:29:04.361874 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:04.361885 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:04.361957 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:04.362230 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:04.362236 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:04.362254 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:04.362651 (+ 397us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:29:04.362652 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:04.362660 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:29:04.362683 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:04.363027 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:04.363029 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:04.363034 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 14:29:04.363145 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 14:29:04.363241 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":55,"thread_start_us":133,"threads_started":1}
I20260808 14:29:04.364117 12659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0db305108a7b406bbff72738439082f7" candidate_uuid: "b249c9a397ef47d9b5a9084032aea595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431c460ae7dd4d32962a5f90ffffc0a6" is_pre_election: true
I20260808 14:29:04.364274 12935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0db305108a7b406bbff72738439082f7" candidate_uuid: "b249c9a397ef47d9b5a9084032aea595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5d08b3e408542239645ed925ebcb9eb" is_pre_election: true
I20260808 14:29:04.364473 12659 raft_consensus.cc:2454] T 0db305108a7b406bbff72738439082f7 P 431c460ae7dd4d32962a5f90ffffc0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b249c9a397ef47d9b5a9084032aea595 in term 0.
I20260808 14:29:04.364627 12935 raft_consensus.cc:2454] T 0db305108a7b406bbff72738439082f7 P a5d08b3e408542239645ed925ebcb9eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b249c9a397ef47d9b5a9084032aea595 in term 0.
I20260808 14:29:04.365248 12729 leader_election.cc:302] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 431c460ae7dd4d32962a5f90ffffc0a6, b249c9a397ef47d9b5a9084032aea595; no voters:
I20260808 14:29:04.365579 13014 raft_consensus.cc:2781] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:29:04.365679 13014 raft_consensus.cc:482] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:29:04.365739 13014 raft_consensus.cc:3037] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:04.366812 13014 raft_consensus.cc:504] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b249c9a397ef47d9b5a9084032aea595" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38159 } } peers { permanent_uuid: "a5d08b3e408542239645ed925ebcb9eb" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37161 } } peers { permanent_uuid: "431c460ae7dd4d32962a5f90ffffc0a6" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 36761 } }
I20260808 14:29:04.367195 13014 leader_election.cc:288] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595 [CANDIDATE]: Term 1 election: Requested vote from peers a5d08b3e408542239645ed925ebcb9eb (127.7.125.67:37161), 431c460ae7dd4d32962a5f90ffffc0a6 (127.7.125.65:36761)
I20260808 14:29:04.367547 12935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0db305108a7b406bbff72738439082f7" candidate_uuid: "b249c9a397ef47d9b5a9084032aea595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5d08b3e408542239645ed925ebcb9eb"
I20260808 14:29:04.367575 12659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0db305108a7b406bbff72738439082f7" candidate_uuid: "b249c9a397ef47d9b5a9084032aea595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431c460ae7dd4d32962a5f90ffffc0a6"
I20260808 14:29:04.367672 12935 raft_consensus.cc:3037] T 0db305108a7b406bbff72738439082f7 P a5d08b3e408542239645ed925ebcb9eb [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:04.367723 12659 raft_consensus.cc:3037] T 0db305108a7b406bbff72738439082f7 P 431c460ae7dd4d32962a5f90ffffc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:04.368968 12659 raft_consensus.cc:2454] T 0db305108a7b406bbff72738439082f7 P 431c460ae7dd4d32962a5f90ffffc0a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b249c9a397ef47d9b5a9084032aea595 in term 1.
I20260808 14:29:04.369154 12935 raft_consensus.cc:2454] T 0db305108a7b406bbff72738439082f7 P a5d08b3e408542239645ed925ebcb9eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b249c9a397ef47d9b5a9084032aea595 in term 1.
I20260808 14:29:04.369310 12729 leader_election.cc:302] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 431c460ae7dd4d32962a5f90ffffc0a6, b249c9a397ef47d9b5a9084032aea595; no voters:
I20260808 14:29:04.369506 13014 raft_consensus.cc:2781] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:29:04.369738 13014 raft_consensus.cc:686] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595 [term 1 LEADER]: Becoming Leader. State: Replica: b249c9a397ef47d9b5a9084032aea595, State: Running, Role: LEADER
I20260808 14:29:04.370062 13014 consensus_queue.cc:237] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b249c9a397ef47d9b5a9084032aea595" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38159 } } peers { permanent_uuid: "a5d08b3e408542239645ed925ebcb9eb" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37161 } } peers { permanent_uuid: "431c460ae7dd4d32962a5f90ffffc0a6" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 36761 } }
I20260808 14:29:04.373399 12507 catalog_manager.cc:5495] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595 reported cstate change: term changed from 0 to 1, leader changed from <none> to b249c9a397ef47d9b5a9084032aea595 (127.7.125.66). New cstate: current_term: 1 leader_uuid: "b249c9a397ef47d9b5a9084032aea595" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b249c9a397ef47d9b5a9084032aea595" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5d08b3e408542239645ed925ebcb9eb" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "431c460ae7dd4d32962a5f90ffffc0a6" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 36761 } health_report { overall_health: UNKNOWN } } }
I20260808 14:29:04.411888 13001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:04.410400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49390 (local address 127.7.125.66:38159)
0808 14:29:04.410534 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:04.410538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:04.410550 (+ 12us) server_negotiation.cc:407] Connection header received
0808 14:29:04.410631 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:04.410633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:04.410692 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:04.410765 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:04.411527 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:04.411530 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:04.411532 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:29:04.411542 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:04.411561 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:29:04.411569 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:04.411645 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:04.411730 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 14:29:04.411767 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 14:29:04.418006 12659 raft_consensus.cc:1261] T 0db305108a7b406bbff72738439082f7 P 431c460ae7dd4d32962a5f90ffffc0a6 [term 1 FOLLOWER]: Refusing update from remote peer b249c9a397ef47d9b5a9084032aea595: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:29:04.418006 12935 raft_consensus.cc:1261] T 0db305108a7b406bbff72738439082f7 P a5d08b3e408542239645ed925ebcb9eb [term 1 FOLLOWER]: Refusing update from remote peer b249c9a397ef47d9b5a9084032aea595: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:29:04.418776 13016 consensus_queue.cc:1035] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5d08b3e408542239645ed925ebcb9eb" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:29:04.418995 13014 consensus_queue.cc:1035] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595 [LEADER]: Connected to new peer: Peer: permanent_uuid: "431c460ae7dd4d32962a5f90ffffc0a6" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 36761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 14:29:04.426218 12989 tablet.cc:2369] T 0db305108a7b406bbff72738439082f7 P a5d08b3e408542239645ed925ebcb9eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:29:04.427402 13019 mvcc.cc:205] Tried to move back new op lower bound from 7316272514728607744 to 7316272514542170112. Current Snapshot: MvccSnapshot[applied={T|T < 7316272514728607744}]
I20260808 14:29:04.428733 13020 mvcc.cc:205] Tried to move back new op lower bound from 7316272514728607744 to 7316272514542170112. Current Snapshot: MvccSnapshot[applied={T|T < 7316272514728607744 or (T in {7316272514728607744})}]
I20260808 14:29:04.437222 12507 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:29:04.437417 12507 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:29:04.439950 12507 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2bdbb47ea69e4863bfdaa025fd4c84d3: Sending DeleteTablet for 3 replicas of tablet 0db305108a7b406bbff72738439082f7
I20260808 14:29:04.440816 12777 tablet_service.cc:1514] Processing DeleteTablet for tablet 0db305108a7b406bbff72738439082f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:04 UTC) from {username='slave'} at 127.0.0.1:49384
I20260808 14:29:04.440987 12638 tablet_service.cc:1514] Processing DeleteTablet for tablet 0db305108a7b406bbff72738439082f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:04 UTC) from {username='slave'} at 127.0.0.1:44126
I20260808 14:29:04.441351 12913 tablet_service.cc:1514] Processing DeleteTablet for tablet 0db305108a7b406bbff72738439082f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:04 UTC) from {username='slave'} at 127.0.0.1:56054
I20260808 14:29:04.441562 13030 tablet_replica.cc:330] T 0db305108a7b406bbff72738439082f7 P 431c460ae7dd4d32962a5f90ffffc0a6: stopping tablet replica
I20260808 14:29:04.441829 13029 tablet_replica.cc:330] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595: stopping tablet replica
I20260808 14:29:04.441836 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 12570
I20260808 14:29:04.443064 13029 raft_consensus.cc:2229] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:29:04.443423 13029 raft_consensus.cc:2258] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:04.447150 13029 ts_tablet_manager.cc:1905] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:29:04.449208 13031 tablet_replica.cc:330] T 0db305108a7b406bbff72738439082f7 P a5d08b3e408542239645ed925ebcb9eb: stopping tablet replica
I20260808 14:29:04.449642 13031 raft_consensus.cc:2229] T 0db305108a7b406bbff72738439082f7 P a5d08b3e408542239645ed925ebcb9eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:29:04.450026 13029 ts_tablet_manager.cc:1918] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:29:04.450130 13029 log.cc:1196] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/0db305108a7b406bbff72738439082f7
I20260808 14:29:04.450161 13031 raft_consensus.cc:2258] T 0db305108a7b406bbff72738439082f7 P a5d08b3e408542239645ed925ebcb9eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:04.450469 13029 ts_tablet_manager.cc:1939] T 0db305108a7b406bbff72738439082f7 P b249c9a397ef47d9b5a9084032aea595: Deleting consensus metadata
I20260808 14:29:04.451776 12491 catalog_manager.cc:4841] TS b249c9a397ef47d9b5a9084032aea595 (127.7.125.66:38159): tablet 0db305108a7b406bbff72738439082f7 (table test-table [id=8dc7cdd7a1ce45f18ee9d131031632e5]) successfully deleted
I20260808 14:29:04.452093 13031 ts_tablet_manager.cc:1905] T 0db305108a7b406bbff72738439082f7 P a5d08b3e408542239645ed925ebcb9eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:29:04.455549 13031 ts_tablet_manager.cc:1918] T 0db305108a7b406bbff72738439082f7 P a5d08b3e408542239645ed925ebcb9eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:29:04.455654 13031 log.cc:1196] T 0db305108a7b406bbff72738439082f7 P a5d08b3e408542239645ed925ebcb9eb: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/0db305108a7b406bbff72738439082f7
W20260808 14:29:04.455740 12491 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.125.65:36761 (error 108)
I20260808 14:29:04.455983 13031 ts_tablet_manager.cc:1939] T 0db305108a7b406bbff72738439082f7 P a5d08b3e408542239645ed925ebcb9eb: Deleting consensus metadata
I20260808 14:29:04.456619 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 12713
I20260808 14:29:04.456909 13000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:04.456561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:36761 (local address 127.0.0.1:44128)
0808 14:29:04.456708 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:04.456800 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.65:36761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 14:29:04.460605 12492 catalog_manager.cc:4841] TS a5d08b3e408542239645ed925ebcb9eb (127.7.125.67:37161): tablet 0db305108a7b406bbff72738439082f7 (table test-table [id=8dc7cdd7a1ce45f18ee9d131031632e5]) successfully deleted
W20260808 14:29:04.462100 12491 catalog_manager.cc:4584] TS 431c460ae7dd4d32962a5f90ffffc0a6 (127.7.125.65:36761): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0db305108a7b406bbff72738439082f7: Network error: Client connection negotiation failed: client connection to 127.7.125.65:36761: connect: Connection refused (error 111)
I20260808 14:29:04.467075 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 12851
I20260808 14:29:04.473520 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 12474
2026-08-08T14:29:04Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1109 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 14:29:04.496606 7669 mini_oidc.cc:152] Starting JWKS server
I20260808 14:29:04.498292 7669 webserver.cc:460] Webserver started at https://127.0.0.1:34741/ using document root <none> and password file <none>
I20260808 14:29:04.498441 7669 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 14:29:04.498664 7669 webserver.cc:460] Webserver started at http://127.0.0.1:38247/ using document root <none> and password file <none>
2026-08-08T14:29:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:29:04Z Disabled control of system clock
I20260808 14:29:04.509061 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:36063
--webserver_interface=127.7.125.126
--webserver_port=0
--builtin_ntp_servers=127.7.125.84:37325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:36063
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34741/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199316863421-7669-0/testchainca.cert with env {}
W20260808 14:29:04.618909 13039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:04.619252 13039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:04.619343 13039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:04.623384 13039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:29:04.623595 13039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:04.623714 13039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:04.623780 13039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:29:04.623826 13039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 14:29:04.623867 13039 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 14:29:04.623903 13039 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34741/jwks.json
I20260808 14:29:04.630342 13039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:37325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:36063
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:36063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34741/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199316863421-7669-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:04.631572 13039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:04.632721 13039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:04.641819 13047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:04.641825 13045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:04.641856 13044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:04.642138 13039 server_base.cc:1034] running on GCE node
I20260808 14:29:04.643093 13039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:04.644276 13039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:04.645444 13039 hybrid_clock.cc:648] HybridClock initialized: now 1786199344645415 us; error 42 us; skew 500 ppm
I20260808 14:29:04.647485 13039 webserver.cc:460] Webserver started at http://127.7.125.126:34153/ using document root <none> and password file <none>
I20260808 14:29:04.648073 13039 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:04.648135 13039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:04.648350 13039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:04.650486 13039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "4ffc2e56aa574696a36351af82817396"
format_stamp: "Formatted at 2026-08-08 14:29:04 on dist-test-slave-w206"
server_key: "1a56e8c65a4938dc0b2c6077d71c2bc8"
server_key_iv: "5b0e697239b41c42b0a4887b4b4d06c4"
server_key_version: "encryptionkey@0"
I20260808 14:29:04.651165 13039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "4ffc2e56aa574696a36351af82817396"
format_stamp: "Formatted at 2026-08-08 14:29:04 on dist-test-slave-w206"
server_key: "1a56e8c65a4938dc0b2c6077d71c2bc8"
server_key_iv: "5b0e697239b41c42b0a4887b4b4d06c4"
server_key_version: "encryptionkey@0"
I20260808 14:29:04.655064 13039 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 14:29:04.657541 13053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:04.658802 13039 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 14:29:04.658947 13039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "4ffc2e56aa574696a36351af82817396"
format_stamp: "Formatted at 2026-08-08 14:29:04 on dist-test-slave-w206"
server_key: "1a56e8c65a4938dc0b2c6077d71c2bc8"
server_key_iv: "5b0e697239b41c42b0a4887b4b4d06c4"
server_key_version: "encryptionkey@0"
I20260808 14:29:04.659127 13039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:04.672710 13039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:04.673431 13039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:04.673641 13039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:04.680805 13110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:36063 every 8 connection(s)
I20260808 14:29:04.680814 13039 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:36063
I20260808 14:29:04.681830 13039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:29:04.684515 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 13039
I20260808 14:29:04.684669 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:29:04.684742 13111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:04.684998 7669 external_mini_cluster.cc:1428] Setting key 307cc2ec706312f621064a5dfd3601e2
I20260808 14:29:04.692384 13111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ffc2e56aa574696a36351af82817396: Bootstrap starting.
I20260808 14:29:04.695149 13111 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ffc2e56aa574696a36351af82817396: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:04.695591 13114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:04.686314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47282 (local address 127.7.125.126:36063)
0808 14:29:04.687035 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:04.687054 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:04.687099 (+ 45us) server_negotiation.cc:407] Connection header received
0808 14:29:04.687829 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:04.687855 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:04.688383 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:04.688753 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:04.689213 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:04.690650 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:04.691664 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:04.692338 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:04.692504 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:04.693411 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:04.693421 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:04.693441 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:29:04.693986 (+ 545us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:04.694055 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:29:04.694081 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:04.694179 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:04.694279 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 14:29:04.694545 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":121,"threads_started":1}
I20260808 14:29:04.696061 13111 log.cc:824] T 00000000000000000000000000000000 P 4ffc2e56aa574696a36351af82817396: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:04.698294 13111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ffc2e56aa574696a36351af82817396: No bootstrap required, opened a new log
I20260808 14:29:04.698557 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:29:04.702214 13111 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ffc2e56aa574696a36351af82817396 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ffc2e56aa574696a36351af82817396" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36063 } }
I20260808 14:29:04.702435 13111 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ffc2e56aa574696a36351af82817396 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:04.702519 13111 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ffc2e56aa574696a36351af82817396 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ffc2e56aa574696a36351af82817396, State: Initialized, Role: FOLLOWER
I20260808 14:29:04.702955 13111 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ffc2e56aa574696a36351af82817396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ffc2e56aa574696a36351af82817396" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36063 } }
I20260808 14:29:04.703145 13111 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ffc2e56aa574696a36351af82817396 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:29:04.703210 13111 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ffc2e56aa574696a36351af82817396 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:29:04.703333 13111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ffc2e56aa574696a36351af82817396 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:04.704226 13111 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ffc2e56aa574696a36351af82817396 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ffc2e56aa574696a36351af82817396" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36063 } }
I20260808 14:29:04.704555 13111 leader_election.cc:302] T 00000000000000000000000000000000 P 4ffc2e56aa574696a36351af82817396 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ffc2e56aa574696a36351af82817396; no voters:
I20260808 14:29:04.704928 13111 leader_election.cc:288] T 00000000000000000000000000000000 P 4ffc2e56aa574696a36351af82817396 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:29:04.705021 13118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ffc2e56aa574696a36351af82817396 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:29:04.705224 13118 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ffc2e56aa574696a36351af82817396 [term 1 LEADER]: Becoming Leader. State: Replica: 4ffc2e56aa574696a36351af82817396, State: Running, Role: LEADER
I20260808 14:29:04.705545 13118 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ffc2e56aa574696a36351af82817396 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ffc2e56aa574696a36351af82817396" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36063 } }
I20260808 14:29:04.706238 13111 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ffc2e56aa574696a36351af82817396 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:29:04.707299 13120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ffc2e56aa574696a36351af82817396 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ffc2e56aa574696a36351af82817396" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ffc2e56aa574696a36351af82817396" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36063 } } }
I20260808 14:29:04.707480 13120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ffc2e56aa574696a36351af82817396 [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:04.707914 13127 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:29:04.708261 13119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ffc2e56aa574696a36351af82817396 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ffc2e56aa574696a36351af82817396. Latest consensus state: current_term: 1 leader_uuid: "4ffc2e56aa574696a36351af82817396" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ffc2e56aa574696a36351af82817396" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36063 } } }
I20260808 14:29:04.708372 13119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ffc2e56aa574696a36351af82817396 [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:04.710633 13127 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:29:04.716179 13127 catalog_manager.cc:1347] Generated new cluster ID: e7bb9f6dc6084d3da05bc0d46a993084
I20260808 14:29:04.716252 13127 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:29:04.726367 13127 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:29:04.727475 13127 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:29:04.736570 13127 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ffc2e56aa574696a36351af82817396: Generated new TSK 0
I20260808 14:29:04.737406 13127 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 14:29:04.767081 13143 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 14:29:04.767310 13143 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 14:29:04.767987 13114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:04.749372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47296 (local address 127.7.125.126:36063)
0808 14:29:04.749504 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:04.749509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:04.749709 (+ 200us) server_negotiation.cc:407] Connection header received
0808 14:29:04.749974 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:04.749977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:04.750026 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:04.750133 (+ 107us) server_negotiation.cc:226] Negotiated authn=JWT
0808 14:29:04.750418 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:04.750955 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:04.751879 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:04.752105 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:04.752366 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:04.752840 (+ 474us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 14:29:04.767456 (+ 14616us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34741/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 14:29:04.767824 (+ 368us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47296: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34741/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 14:29:04.768066 13114 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47296: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34741/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 14:29:04.769419 13142 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:04.749264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36063 (local address 127.0.0.1:47296)
0808 14:29:04.749622 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:04.749634 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:04.749792 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:04.750151 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:04.750155 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:04.750163 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 14:29:04.750298 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:04.750304 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:04.751118 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:04.751122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:04.751749 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:04.751757 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:04.752382 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:04.752386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:04.752504 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:04.752586 (+ 82us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 14:29:04.767757 (+ 15171us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34741/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 14:29:04.768033 (+ 276us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.125.126:36063: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34741/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":112,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.7.125.126:36063: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34741/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 14:29:04.770241 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 13039
2026-08-08T14:29:04Z chronyd exiting
I20260808 14:29:04.809978 7669 test_util.cc:175] -----------------------------------------------
I20260808 14:29:04.810082 7669 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199316863421-7669-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (315 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 14:29:04.811244 7669 mini_oidc.cc:152] Starting JWKS server
I20260808 14:29:04.812101 7669 webserver.cc:460] Webserver started at https://127.0.0.1:44635/ using document root <none> and password file <none>
I20260808 14:29:04.812180 7669 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 14:29:04.812389 7669 webserver.cc:460] Webserver started at http://127.0.0.1:34261/ using document root <none> and password file <none>
2026-08-08T14:29:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:29:04Z Disabled control of system clock
I20260808 14:29:04.823391 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:44225
--webserver_interface=127.7.125.126
--webserver_port=0
--builtin_ntp_servers=127.7.125.84:43819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:44225
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44635/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199316863421-7669-0/testchainca.cert with env {}
W20260808 14:29:04.935448 13149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:04.935729 13149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:04.935770 13149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:04.939844 13149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:29:04.939908 13149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:04.939941 13149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:04.939963 13149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:29:04.939985 13149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 14:29:04.940001 13149 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 14:29:04.940013 13149 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44635/jwks.json
I20260808 14:29:04.945436 13149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:43819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:44225
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:44225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44635/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199316863421-7669-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:04.946488 13149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:04.947480 13149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:04.956416 13155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:04.956494 13157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:04.956411 13154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:04.956784 13149 server_base.cc:1034] running on GCE node
I20260808 14:29:04.957404 13149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:04.958470 13149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:04.959654 13149 hybrid_clock.cc:648] HybridClock initialized: now 1786199344959622 us; error 46 us; skew 500 ppm
I20260808 14:29:04.961465 13149 webserver.cc:460] Webserver started at http://127.7.125.126:36761/ using document root <none> and password file <none>
I20260808 14:29:04.962034 13149 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:04.962127 13149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:04.962359 13149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:04.964525 13149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "1d87eb90b3e94490bb08e8a63fbf7fe0"
format_stamp: "Formatted at 2026-08-08 14:29:04 on dist-test-slave-w206"
server_key: "75739b0acf31b039c1bb27af1f05b9c5"
server_key_iv: "a66cde57cf8e22f47f1691dfbb468c8a"
server_key_version: "encryptionkey@0"
I20260808 14:29:04.965122 13149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "1d87eb90b3e94490bb08e8a63fbf7fe0"
format_stamp: "Formatted at 2026-08-08 14:29:04 on dist-test-slave-w206"
server_key: "75739b0acf31b039c1bb27af1f05b9c5"
server_key_iv: "a66cde57cf8e22f47f1691dfbb468c8a"
server_key_version: "encryptionkey@0"
I20260808 14:29:04.968912 13149 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 14:29:04.971359 13163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:04.972606 13149 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:29:04.972748 13149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "1d87eb90b3e94490bb08e8a63fbf7fe0"
format_stamp: "Formatted at 2026-08-08 14:29:04 on dist-test-slave-w206"
server_key: "75739b0acf31b039c1bb27af1f05b9c5"
server_key_iv: "a66cde57cf8e22f47f1691dfbb468c8a"
server_key_version: "encryptionkey@0"
I20260808 14:29:04.972868 13149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:04.984922 13149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:04.985625 13149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:04.985832 13149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:04.992986 13149 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:44225
I20260808 14:29:04.993009 13219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:44225 every 8 connection(s)
I20260808 14:29:04.993907 13149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:29:04.996950 13220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:04.999883 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 13149
I20260808 14:29:05.000033 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:29:05.000355 7669 external_mini_cluster.cc:1428] Setting key 5f59b120e51b9a13eb910d85352f93ef
I20260808 14:29:05.003484 13220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d87eb90b3e94490bb08e8a63fbf7fe0: Bootstrap starting.
I20260808 14:29:05.005797 13220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d87eb90b3e94490bb08e8a63fbf7fe0: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:05.006563 13220 log.cc:824] T 00000000000000000000000000000000 P 1d87eb90b3e94490bb08e8a63fbf7fe0: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:05.008500 13220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d87eb90b3e94490bb08e8a63fbf7fe0: No bootstrap required, opened a new log
I20260808 14:29:05.009640 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:05.001751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50516 (local address 127.7.125.126:44225)
0808 14:29:05.002417 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:05.002432 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:05.002467 (+ 35us) server_negotiation.cc:407] Connection header received
0808 14:29:05.003149 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:05.003167 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:05.003570 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:05.003956 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:05.004406 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:05.005357 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:05.006325 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:05.006858 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:05.007064 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:05.007939 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:05.007952 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:05.007967 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:29:05.008356 (+ 389us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:05.008417 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:29:05.008438 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:05.008551 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:05.008697 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 14:29:05.008956 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":138,"threads_started":1}
I20260808 14:29:05.012581 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:29:05.012815 13220 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d87eb90b3e94490bb08e8a63fbf7fe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d87eb90b3e94490bb08e8a63fbf7fe0" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44225 } }
I20260808 14:29:05.013074 13220 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d87eb90b3e94490bb08e8a63fbf7fe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:05.013175 13220 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d87eb90b3e94490bb08e8a63fbf7fe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d87eb90b3e94490bb08e8a63fbf7fe0, State: Initialized, Role: FOLLOWER
I20260808 14:29:05.013710 13220 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d87eb90b3e94490bb08e8a63fbf7fe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d87eb90b3e94490bb08e8a63fbf7fe0" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44225 } }
I20260808 14:29:05.013885 13220 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d87eb90b3e94490bb08e8a63fbf7fe0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:29:05.013967 13220 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d87eb90b3e94490bb08e8a63fbf7fe0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:29:05.014066 13220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d87eb90b3e94490bb08e8a63fbf7fe0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:05.015118 13220 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d87eb90b3e94490bb08e8a63fbf7fe0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d87eb90b3e94490bb08e8a63fbf7fe0" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44225 } }
I20260808 14:29:05.015460 13220 leader_election.cc:302] T 00000000000000000000000000000000 P 1d87eb90b3e94490bb08e8a63fbf7fe0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d87eb90b3e94490bb08e8a63fbf7fe0; no voters:
I20260808 14:29:05.015769 13220 leader_election.cc:288] T 00000000000000000000000000000000 P 1d87eb90b3e94490bb08e8a63fbf7fe0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:29:05.015888 13225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d87eb90b3e94490bb08e8a63fbf7fe0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:29:05.016129 13225 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d87eb90b3e94490bb08e8a63fbf7fe0 [term 1 LEADER]: Becoming Leader. State: Replica: 1d87eb90b3e94490bb08e8a63fbf7fe0, State: Running, Role: LEADER
I20260808 14:29:05.016485 13225 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d87eb90b3e94490bb08e8a63fbf7fe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d87eb90b3e94490bb08e8a63fbf7fe0" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44225 } }
I20260808 14:29:05.017025 13220 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d87eb90b3e94490bb08e8a63fbf7fe0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:29:05.018077 13227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d87eb90b3e94490bb08e8a63fbf7fe0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d87eb90b3e94490bb08e8a63fbf7fe0. Latest consensus state: current_term: 1 leader_uuid: "1d87eb90b3e94490bb08e8a63fbf7fe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d87eb90b3e94490bb08e8a63fbf7fe0" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44225 } } }
I20260808 14:29:05.018221 13227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d87eb90b3e94490bb08e8a63fbf7fe0 [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:05.018586 13226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d87eb90b3e94490bb08e8a63fbf7fe0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d87eb90b3e94490bb08e8a63fbf7fe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d87eb90b3e94490bb08e8a63fbf7fe0" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44225 } } }
I20260808 14:29:05.019241 13226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d87eb90b3e94490bb08e8a63fbf7fe0 [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:05.019425 13235 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:29:05.022588 13235 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:29:05.028434 13235 catalog_manager.cc:1347] Generated new cluster ID: 79fcf885f4d6475cb32435eff67834fc
I20260808 14:29:05.028517 13235 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:29:05.035456 13235 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:29:05.036298 13235 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:29:05.043931 13235 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d87eb90b3e94490bb08e8a63fbf7fe0: Generated new TSK 0
I20260808 14:29:05.044600 13235 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 14:29:05.079514 13251 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 14:29:05.079610 13251 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 14:29:05.080235 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:05.056754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50522 (local address 127.7.125.126:44225)
0808 14:29:05.056905 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:05.056909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:05.057087 (+ 178us) server_negotiation.cc:407] Connection header received
0808 14:29:05.057248 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:05.057250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:05.057298 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:05.057407 (+ 109us) server_negotiation.cc:226] Negotiated authn=JWT
0808 14:29:05.057661 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:05.058200 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:05.059463 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:05.059698 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:05.059778 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:05.060131 (+ 353us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 14:29:05.079854 (+ 19723us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44635/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 14:29:05.080116 (+ 262us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50522: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44635/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 14:29:05.080323 13223 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50522: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44635/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 14:29:05.080453 13250 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:05.056683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:44225 (local address 127.0.0.1:50522)
0808 14:29:05.056988 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:05.057004 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:05.057119 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:05.057424 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:05.057427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:05.057435 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 14:29:05.057548 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:05.057554 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:05.058319 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:05.058322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:05.059343 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:05.059354 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:05.059827 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:05.059831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:05.059935 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:05.059963 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 14:29:05.080078 (+ 20115us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44635/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 14:29:05.080262 (+ 184us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.125.126:44225: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44635/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":118,"threads_started":1}
I20260808 14:29:05.081204 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 13149
2026-08-08T14:29:05Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (301 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 14:29:05.112906 7669 mini_oidc.cc:152] Starting JWKS server
I20260808 14:29:05.113803 7669 webserver.cc:460] Webserver started at https://127.0.0.1:36409/ using document root <none> and password file <none>
I20260808 14:29:05.113947 7669 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 14:29:05.114228 7669 webserver.cc:460] Webserver started at http://127.0.0.1:36999/ using document root <none> and password file <none>
2026-08-08T14:29:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:29:05Z Disabled control of system clock
I20260808 14:29:05.124897 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:44137
--webserver_interface=127.7.125.126
--webserver_port=0
--builtin_ntp_servers=127.7.125.84:41573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:44137
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36409/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 14:29:05.236822 13257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:05.237089 13257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:05.237128 13257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:05.241110 13257 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:29:05.241166 13257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:05.241182 13257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:05.241199 13257 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:29:05.241214 13257 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 14:29:05.241230 13257 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 14:29:05.241319 13257 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36409/jwks.json
I20260808 14:29:05.246031 13257 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:41573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:44137
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:44137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36409/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:05.247119 13257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:05.248517 13257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:05.257247 13263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:05.257243 13265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:05.257308 13262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:05.257603 13257 server_base.cc:1034] running on GCE node
I20260808 14:29:05.258155 13257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:05.259327 13257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:05.260486 13257 hybrid_clock.cc:648] HybridClock initialized: now 1786199345260461 us; error 42 us; skew 500 ppm
I20260808 14:29:05.262312 13257 webserver.cc:460] Webserver started at http://127.7.125.126:36921/ using document root <none> and password file <none>
I20260808 14:29:05.262897 13257 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:05.262959 13257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:05.263228 13257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:05.265318 13257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "e4ea91a42271479abf41acc4de05cf9f"
format_stamp: "Formatted at 2026-08-08 14:29:05 on dist-test-slave-w206"
server_key: "ed66d4b23c37e3128f8e80827e0f9bfc"
server_key_iv: "7237de5d4e4b58e593f9388829e4ab20"
server_key_version: "encryptionkey@0"
I20260808 14:29:05.265893 13257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "e4ea91a42271479abf41acc4de05cf9f"
format_stamp: "Formatted at 2026-08-08 14:29:05 on dist-test-slave-w206"
server_key: "ed66d4b23c37e3128f8e80827e0f9bfc"
server_key_iv: "7237de5d4e4b58e593f9388829e4ab20"
server_key_version: "encryptionkey@0"
I20260808 14:29:05.269688 13257 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260808 14:29:05.272176 13273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:05.273356 13257 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:29:05.273481 13257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "e4ea91a42271479abf41acc4de05cf9f"
format_stamp: "Formatted at 2026-08-08 14:29:05 on dist-test-slave-w206"
server_key: "ed66d4b23c37e3128f8e80827e0f9bfc"
server_key_iv: "7237de5d4e4b58e593f9388829e4ab20"
server_key_version: "encryptionkey@0"
I20260808 14:29:05.273583 13257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:05.289191 13257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:05.289930 13257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:05.290138 13257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:05.297078 13257 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:44137
I20260808 14:29:05.297086 13326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:44137 every 8 connection(s)
I20260808 14:29:05.298048 13257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:29:05.300856 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 13257
I20260808 14:29:05.301002 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:29:05.301322 7669 external_mini_cluster.cc:1428] Setting key c74cfe98161dc938a5a4aaa85425b1d6
I20260808 14:29:05.301481 13327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:05.308082 13327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4ea91a42271479abf41acc4de05cf9f: Bootstrap starting.
I20260808 14:29:05.310388 13330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:05.302624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46450 (local address 127.7.125.126:44137)
0808 14:29:05.303285 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:05.303301 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:05.303355 (+ 54us) server_negotiation.cc:407] Connection header received
0808 14:29:05.303986 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:05.304002 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:05.304393 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:05.304723 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:05.305152 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:05.306121 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:05.307083 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:05.307558 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:05.307734 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:05.308651 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:05.308656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:05.308668 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:29:05.309044 (+ 376us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:05.309096 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:29:05.309115 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:05.309193 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:05.309395 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 14:29:05.309682 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":122,"threads_started":1}
I20260808 14:29:05.310729 13327 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4ea91a42271479abf41acc4de05cf9f: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:05.311621 13327 log.cc:824] T 00000000000000000000000000000000 P e4ea91a42271479abf41acc4de05cf9f: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:05.313463 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:29:05.313908 13327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4ea91a42271479abf41acc4de05cf9f: No bootstrap required, opened a new log
I20260808 14:29:05.317577 13327 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4ea91a42271479abf41acc4de05cf9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ea91a42271479abf41acc4de05cf9f" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44137 } }
I20260808 14:29:05.317795 13327 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4ea91a42271479abf41acc4de05cf9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:05.317858 13327 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4ea91a42271479abf41acc4de05cf9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4ea91a42271479abf41acc4de05cf9f, State: Initialized, Role: FOLLOWER
I20260808 14:29:05.318285 13327 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4ea91a42271479abf41acc4de05cf9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ea91a42271479abf41acc4de05cf9f" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44137 } }
I20260808 14:29:05.318434 13327 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4ea91a42271479abf41acc4de05cf9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:29:05.318499 13327 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4ea91a42271479abf41acc4de05cf9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:29:05.318598 13327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4ea91a42271479abf41acc4de05cf9f [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:05.319585 13327 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4ea91a42271479abf41acc4de05cf9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ea91a42271479abf41acc4de05cf9f" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44137 } }
I20260808 14:29:05.319933 13327 leader_election.cc:302] T 00000000000000000000000000000000 P e4ea91a42271479abf41acc4de05cf9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4ea91a42271479abf41acc4de05cf9f; no voters:
I20260808 14:29:05.320237 13327 leader_election.cc:288] T 00000000000000000000000000000000 P e4ea91a42271479abf41acc4de05cf9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:29:05.320379 13332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4ea91a42271479abf41acc4de05cf9f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:29:05.320878 13332 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4ea91a42271479abf41acc4de05cf9f [term 1 LEADER]: Becoming Leader. State: Replica: e4ea91a42271479abf41acc4de05cf9f, State: Running, Role: LEADER
I20260808 14:29:05.321192 13332 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4ea91a42271479abf41acc4de05cf9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ea91a42271479abf41acc4de05cf9f" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44137 } }
I20260808 14:29:05.321525 13327 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4ea91a42271479abf41acc4de05cf9f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:29:05.322892 13333 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4ea91a42271479abf41acc4de05cf9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4ea91a42271479abf41acc4de05cf9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ea91a42271479abf41acc4de05cf9f" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44137 } } }
I20260808 14:29:05.323048 13333 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4ea91a42271479abf41acc4de05cf9f [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:05.322962 13334 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4ea91a42271479abf41acc4de05cf9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4ea91a42271479abf41acc4de05cf9f. Latest consensus state: current_term: 1 leader_uuid: "e4ea91a42271479abf41acc4de05cf9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ea91a42271479abf41acc4de05cf9f" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44137 } } }
I20260808 14:29:05.323099 13334 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4ea91a42271479abf41acc4de05cf9f [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:05.323414 13342 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:29:05.326190 13342 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:29:05.331813 13342 catalog_manager.cc:1347] Generated new cluster ID: e327072cd1b443afbe185708fc2b6041
I20260808 14:29:05.331892 13342 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:29:05.365929 13342 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:29:05.367153 13342 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:29:05.374053 13342 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4ea91a42271479abf41acc4de05cf9f: Generated new TSK 0
I20260808 14:29:05.374809 13342 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:29:05.388512 13330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:05.383227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46462 (local address 127.7.125.126:44137)
0808 14:29:05.383362 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:05.383369 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:05.383571 (+ 202us) server_negotiation.cc:407] Connection header received
0808 14:29:05.383770 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:05.383773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:05.383837 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:05.383951 (+ 114us) server_negotiation.cc:226] Negotiated authn=JWT
0808 14:29:05.384201 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:05.385217 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:05.386131 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:05.386355 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:05.386467 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:05.386804 (+ 337us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 14:29:05.388114 (+ 1310us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36409/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 14:29:05.388364 (+ 250us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46462: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36409/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 14:29:05.388597 13330 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46462: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36409/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 14:29:05.388615 13357 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:05.383177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:44137 (local address 127.0.0.1:46462)
0808 14:29:05.383478 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:05.383490 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:05.383622 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:05.383951 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:05.383955 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:05.383963 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 14:29:05.384082 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:05.384090 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:05.385364 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:05.385367 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:05.385995 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:05.386003 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:05.386486 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:05.386490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:05.386583 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:05.386614 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 14:29:05.388274 (+ 1660us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36409/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 14:29:05.388438 (+ 164us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.125.126:44137: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36409/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":108,"threads_started":1}
I20260808 14:29:05.389288 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 13257
E20260808 14:29:05.403172 13358 webserver.cc:533] Webserver: sslize failed:
E20260808 14:29:05.403258 13358 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T14:29:05Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (318 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T14:29:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:29:05Z Disabled control of system clock
I20260808 14:29:05.441813 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:36473
--webserver_interface=127.7.125.126
--webserver_port=0
--builtin_ntp_servers=127.7.125.84:35419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:36473
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199316863421-7669-0/testchainca.cert with env {}
W20260808 14:29:05.553354 13362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:05.553625 13362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:05.553664 13362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:05.557734 13362 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:29:05.557796 13362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:05.557812 13362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:05.557829 13362 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:29:05.557844 13362 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 14:29:05.557860 13362 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 14:29:05.557880 13362 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 14:29:05.563208 13362 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:35419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:36473
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:36473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199316863421-7669-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:05.564224 13362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:05.565176 13362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:05.573770 13368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:05.573704 13371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:05.573832 13362 server_base.cc:1034] running on GCE node
W20260808 14:29:05.573717 13367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:05.574586 13362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:05.575804 13362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:05.577011 13362 hybrid_clock.cc:648] HybridClock initialized: now 1786199345576978 us; error 42 us; skew 500 ppm
I20260808 14:29:05.578967 13362 webserver.cc:460] Webserver started at http://127.7.125.126:35683/ using document root <none> and password file <none>
I20260808 14:29:05.579604 13362 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:05.579698 13362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:05.579970 13362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:05.582103 13362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "39cb90efd704479a8f1e851c8760d9ec"
format_stamp: "Formatted at 2026-08-08 14:29:05 on dist-test-slave-w206"
server_key: "4b355b7da08f285e9c4c26c8b9fe0469"
server_key_iv: "7b325d53adb7a7918c3164cf9865a4d1"
server_key_version: "encryptionkey@0"
I20260808 14:29:05.582718 13362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "39cb90efd704479a8f1e851c8760d9ec"
format_stamp: "Formatted at 2026-08-08 14:29:05 on dist-test-slave-w206"
server_key: "4b355b7da08f285e9c4c26c8b9fe0469"
server_key_iv: "7b325d53adb7a7918c3164cf9865a4d1"
server_key_version: "encryptionkey@0"
I20260808 14:29:05.587249 13362 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 14:29:05.589797 13377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:05.591109 13362 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:29:05.591243 13362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "39cb90efd704479a8f1e851c8760d9ec"
format_stamp: "Formatted at 2026-08-08 14:29:05 on dist-test-slave-w206"
server_key: "4b355b7da08f285e9c4c26c8b9fe0469"
server_key_iv: "7b325d53adb7a7918c3164cf9865a4d1"
server_key_version: "encryptionkey@0"
I20260808 14:29:05.591369 13362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:05.604724 13362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:05.605450 13362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:05.605644 13362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:05.613144 13362 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:36473
I20260808 14:29:05.613157 13431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:36473 every 8 connection(s)
I20260808 14:29:05.614161 13362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:29:05.617357 13432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:05.617429 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 13362
I20260808 14:29:05.617559 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:29:05.617892 7669 external_mini_cluster.cc:1428] Setting key 611f71578aa50274b6660ce293d42e43
I20260808 14:29:05.623731 13432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39cb90efd704479a8f1e851c8760d9ec: Bootstrap starting.
I20260808 14:29:05.626078 13432 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39cb90efd704479a8f1e851c8760d9ec: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:05.626798 13432 log.cc:824] T 00000000000000000000000000000000 P 39cb90efd704479a8f1e851c8760d9ec: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:05.627048 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:05.619314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51000 (local address 127.7.125.126:36473)
0808 14:29:05.619955 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:05.619969 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:05.620004 (+ 35us) server_negotiation.cc:407] Connection header received
0808 14:29:05.620676 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:05.620693 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:05.621068 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:05.621407 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:05.621808 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:05.622737 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:05.623661 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:05.624173 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:05.624355 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:05.625237 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:05.625242 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:05.625254 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:29:05.625627 (+ 373us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:05.625685 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:29:05.625705 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:05.625811 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:05.625971 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 14:29:05.626256 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":123,"threads_started":1}
I20260808 14:29:05.628814 13432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39cb90efd704479a8f1e851c8760d9ec: No bootstrap required, opened a new log
I20260808 14:29:05.629815 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:29:05.632660 13432 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39cb90efd704479a8f1e851c8760d9ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39cb90efd704479a8f1e851c8760d9ec" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36473 } }
I20260808 14:29:05.632869 13432 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39cb90efd704479a8f1e851c8760d9ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:05.632915 13432 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39cb90efd704479a8f1e851c8760d9ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39cb90efd704479a8f1e851c8760d9ec, State: Initialized, Role: FOLLOWER
I20260808 14:29:05.633308 13432 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39cb90efd704479a8f1e851c8760d9ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39cb90efd704479a8f1e851c8760d9ec" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36473 } }
I20260808 14:29:05.633426 13432 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39cb90efd704479a8f1e851c8760d9ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:29:05.633472 13432 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39cb90efd704479a8f1e851c8760d9ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:29:05.633541 13432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39cb90efd704479a8f1e851c8760d9ec [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:05.634385 13432 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39cb90efd704479a8f1e851c8760d9ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39cb90efd704479a8f1e851c8760d9ec" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36473 } }
I20260808 14:29:05.634665 13432 leader_election.cc:302] T 00000000000000000000000000000000 P 39cb90efd704479a8f1e851c8760d9ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39cb90efd704479a8f1e851c8760d9ec; no voters:
I20260808 14:29:05.634910 13432 leader_election.cc:288] T 00000000000000000000000000000000 P 39cb90efd704479a8f1e851c8760d9ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:29:05.635039 13437 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39cb90efd704479a8f1e851c8760d9ec [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:29:05.635303 13437 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39cb90efd704479a8f1e851c8760d9ec [term 1 LEADER]: Becoming Leader. State: Replica: 39cb90efd704479a8f1e851c8760d9ec, State: Running, Role: LEADER
I20260808 14:29:05.635591 13437 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39cb90efd704479a8f1e851c8760d9ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39cb90efd704479a8f1e851c8760d9ec" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36473 } }
I20260808 14:29:05.636109 13432 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39cb90efd704479a8f1e851c8760d9ec [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:29:05.637198 13438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39cb90efd704479a8f1e851c8760d9ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39cb90efd704479a8f1e851c8760d9ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39cb90efd704479a8f1e851c8760d9ec" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36473 } } }
I20260808 14:29:05.637329 13438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39cb90efd704479a8f1e851c8760d9ec [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:05.637619 13439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39cb90efd704479a8f1e851c8760d9ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39cb90efd704479a8f1e851c8760d9ec. Latest consensus state: current_term: 1 leader_uuid: "39cb90efd704479a8f1e851c8760d9ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39cb90efd704479a8f1e851c8760d9ec" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36473 } } }
I20260808 14:29:05.637734 13439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39cb90efd704479a8f1e851c8760d9ec [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:05.637717 13446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:29:05.641113 13446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:29:05.647233 13446 catalog_manager.cc:1347] Generated new cluster ID: 3aa12669fa3b43318258b2eda4d2bd14
I20260808 14:29:05.647315 13446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:29:05.655740 13446 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:29:05.656916 13446 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:29:05.667299 13446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39cb90efd704479a8f1e851c8760d9ec: Generated new TSK 0
I20260808 14:29:05.668110 13446 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:29:05.745447 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:05.678455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38440 (local address 127.7.125.126:36473)
0808 14:29:05.678590 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:05.678595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:05.678752 (+ 157us) server_negotiation.cc:407] Connection header received
0808 14:29:05.678904 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:05.678907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:05.678965 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:05.679095 (+ 130us) server_negotiation.cc:226] Negotiated authn=JWT
0808 14:29:05.679316 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:05.680196 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:05.681026 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:05.681427 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:05.681535 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:05.681821 (+ 286us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 14:29:05.744941 (+ 63120us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 14:29:05.745281 (+ 340us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38440: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 14:29:05.745539 13435 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38440: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 14:29:05.745564 13463 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:05.678382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36473 (local address 127.0.0.1:38440)
0808 14:29:05.678669 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:05.678682 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:05.678775 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:05.679113 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:05.679115 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:05.679123 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 14:29:05.679214 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:05.679219 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:05.680300 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:05.680302 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:05.680914 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:05.680922 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:05.681554 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:05.681557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:05.681628 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:05.681650 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 14:29:05.745211 (+ 63561us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 14:29:05.745388 (+ 177us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.125.126:36473: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":123,"threads_started":1}
I20260808 14:29:05.746584 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 13362
2026-08-08T14:29:05Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (335 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (129 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (133 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:29:06 dist-test-slave-w206 krb5kdc[13476](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:29:06 dist-test-slave-w206 krb5kdc[13476](info): set up 2 sockets
Aug 08 14:29:06 dist-test-slave-w206 krb5kdc[13476](info): commencing operation
krb5kdc: starting...
W20260808 14:29:07.124110 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:29:07 dist-test-slave-w206 krb5kdc[13476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:29:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:29:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:29:07.256539 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:42711
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:42711
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:07.369166 13492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:07.369491 13492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:07.369577 13492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:07.373662 13492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:29:07.373749 13492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:07.373787 13492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:07.373852 13492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:29:07.373893 13492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:29:07.379184 13492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:42711
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:42711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:07.380273 13492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:07.381299 13492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:07.389883 13497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:07.390036 13492 server_base.cc:1034] running on GCE node
W20260808 14:29:07.390012 13498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:07.389926 13500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:07.390707 13492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:07.391898 13492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:07.393064 13492 hybrid_clock.cc:648] HybridClock initialized: now 1786199347393046 us; error 31 us; skew 500 ppm
Aug 08 14:29:07 dist-test-slave-w206 krb5kdc[13476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199347, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:07.395901 13492 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:29:07.396843 13492 webserver.cc:460] Webserver started at http://127.7.125.126:40555/ using document root <none> and password file <none>
I20260808 14:29:07.397414 13492 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:07.397480 13492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:07.397714 13492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:07.399842 13492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "818d83f67dca4a2996361bd2d4a64d87"
format_stamp: "Formatted at 2026-08-08 14:29:07 on dist-test-slave-w206"
server_key: "06b1825912fbe41ec9d359f0a926bd40"
server_key_iv: "5ead1ca108f7c364d2740b17a61bb04a"
server_key_version: "encryptionkey@0"
I20260808 14:29:07.400416 13492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "818d83f67dca4a2996361bd2d4a64d87"
format_stamp: "Formatted at 2026-08-08 14:29:07 on dist-test-slave-w206"
server_key: "06b1825912fbe41ec9d359f0a926bd40"
server_key_iv: "5ead1ca108f7c364d2740b17a61bb04a"
server_key_version: "encryptionkey@0"
I20260808 14:29:07.404093 13492 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260808 14:29:07.406425 13507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:07.407697 13492 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 14:29:07.407852 13492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "818d83f67dca4a2996361bd2d4a64d87"
format_stamp: "Formatted at 2026-08-08 14:29:07 on dist-test-slave-w206"
server_key: "06b1825912fbe41ec9d359f0a926bd40"
server_key_iv: "5ead1ca108f7c364d2740b17a61bb04a"
server_key_version: "encryptionkey@0"
I20260808 14:29:07.407967 13492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:07.426045 13492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:07.429219 13492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:07.429446 13492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:07.437202 13492 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:42711
I20260808 14:29:07.437218 13563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:42711 every 8 connection(s)
I20260808 14:29:07.438169 13492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:29:07.440992 13564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:07.443461 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 13492
I20260808 14:29:07.443598 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:29:07.443869 7669 external_mini_cluster.cc:1428] Setting key 2c9ba87338d1ce34e3f973da830c976a
I20260808 14:29:07.447333 13564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 818d83f67dca4a2996361bd2d4a64d87: Bootstrap starting.
I20260808 14:29:07.449687 13564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 818d83f67dca4a2996361bd2d4a64d87: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:07.450486 13564 log.cc:824] T 00000000000000000000000000000000 P 818d83f67dca4a2996361bd2d4a64d87: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:29:07 dist-test-slave-w206 krb5kdc[13476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:29:07.452908 13564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 818d83f67dca4a2996361bd2d4a64d87: No bootstrap required, opened a new log
I20260808 14:29:07.457283 13564 raft_consensus.cc:348] T 00000000000000000000000000000000 P 818d83f67dca4a2996361bd2d4a64d87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d83f67dca4a2996361bd2d4a64d87" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 42711 } }
I20260808 14:29:07.457515 13564 raft_consensus.cc:374] T 00000000000000000000000000000000 P 818d83f67dca4a2996361bd2d4a64d87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:07.457592 13564 raft_consensus.cc:729] T 00000000000000000000000000000000 P 818d83f67dca4a2996361bd2d4a64d87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 818d83f67dca4a2996361bd2d4a64d87, State: Initialized, Role: FOLLOWER
I20260808 14:29:07.458082 13564 consensus_queue.cc:260] T 00000000000000000000000000000000 P 818d83f67dca4a2996361bd2d4a64d87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d83f67dca4a2996361bd2d4a64d87" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 42711 } }
I20260808 14:29:07.458140 13567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:07.445268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42272 (local address 127.7.125.126:42711)
0808 14:29:07.445750 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:07.445765 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:07.445806 (+ 41us) server_negotiation.cc:407] Connection header received
0808 14:29:07.446427 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:07.446443 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:07.446755 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:07.447116 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:07.447954 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:07.448917 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:07.449764 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:07.450283 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:07.450465 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:07.453457 (+ 2992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:07.453463 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:07.453475 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:07.453502 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:07.455238 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:07.455735 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:07.455739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:07.455743 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:07.455865 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:07.456154 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:07.456156 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:07.456157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:07.456516 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:07.456664 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:07.456979 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 14:29:07.457283 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":186,"threads_started":1}
I20260808 14:29:07.458263 13564 raft_consensus.cc:388] T 00000000000000000000000000000000 P 818d83f67dca4a2996361bd2d4a64d87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:29:07.458323 13564 raft_consensus.cc:482] T 00000000000000000000000000000000 P 818d83f67dca4a2996361bd2d4a64d87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:29:07.458427 13564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 818d83f67dca4a2996361bd2d4a64d87 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:07.459421 13564 raft_consensus.cc:504] T 00000000000000000000000000000000 P 818d83f67dca4a2996361bd2d4a64d87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d83f67dca4a2996361bd2d4a64d87" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 42711 } }
I20260808 14:29:07.459769 13564 leader_election.cc:302] T 00000000000000000000000000000000 P 818d83f67dca4a2996361bd2d4a64d87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 818d83f67dca4a2996361bd2d4a64d87; no voters:
I20260808 14:29:07.460165 13564 leader_election.cc:288] T 00000000000000000000000000000000 P 818d83f67dca4a2996361bd2d4a64d87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:29:07.460237 13569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 818d83f67dca4a2996361bd2d4a64d87 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:29:07.460446 13569 raft_consensus.cc:686] T 00000000000000000000000000000000 P 818d83f67dca4a2996361bd2d4a64d87 [term 1 LEADER]: Becoming Leader. State: Replica: 818d83f67dca4a2996361bd2d4a64d87, State: Running, Role: LEADER
I20260808 14:29:07.460778 13569 consensus_queue.cc:237] T 00000000000000000000000000000000 P 818d83f67dca4a2996361bd2d4a64d87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d83f67dca4a2996361bd2d4a64d87" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 42711 } }
I20260808 14:29:07.461476 13564 sys_catalog.cc:564] T 00000000000000000000000000000000 P 818d83f67dca4a2996361bd2d4a64d87 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:29:07.462570 13570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 818d83f67dca4a2996361bd2d4a64d87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "818d83f67dca4a2996361bd2d4a64d87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d83f67dca4a2996361bd2d4a64d87" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 42711 } } }
I20260808 14:29:07.462710 13570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 818d83f67dca4a2996361bd2d4a64d87 [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:07.463004 13571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 818d83f67dca4a2996361bd2d4a64d87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 818d83f67dca4a2996361bd2d4a64d87. Latest consensus state: current_term: 1 leader_uuid: "818d83f67dca4a2996361bd2d4a64d87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d83f67dca4a2996361bd2d4a64d87" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 42711 } } }
I20260808 14:29:07.463094 13571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 818d83f67dca4a2996361bd2d4a64d87 [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:07.463304 13578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:29:07.466655 13578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:29:07.472599 13578 catalog_manager.cc:1347] Generated new cluster ID: 9e052177fc384070a95024fdffd9ae91
I20260808 14:29:07.472687 13578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:29:07.493448 13578 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:29:07.494423 13578 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:29:07.501509 13578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 818d83f67dca4a2996361bd2d4a64d87: Generated new TSK 0
I20260808 14:29:07.502204 13578 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.125.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
I20260808 14:29:07.537915 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:42711
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:07.654553 13590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:07.654820 13590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:07.654865 13590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:07.659085 13590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:07.659147 13590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:07.659209 13590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:29:07.664968 13590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:42711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:07.665983 13590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:07.666957 13590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:07.676483 13595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:07.676474 13597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:07.676474 13599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:07.676802 13590 server_base.cc:1034] running on GCE node
I20260808 14:29:07.677205 13590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:07.677865 13590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:07.679106 13590 hybrid_clock.cc:648] HybridClock initialized: now 1786199347679089 us; error 25 us; skew 500 ppm
Aug 08 14:29:07 dist-test-slave-w206 krb5kdc[13476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199347, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:07.687278 13590 init.cc:377] Logged in from keytab as kudu/127.7.125.65@KRBTEST.COM (short username kudu)
I20260808 14:29:07.688266 13590 webserver.cc:460] Webserver started at http://127.7.125.65:34147/ using document root <none> and password file <none>
I20260808 14:29:07.688884 13590 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:07.688984 13590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:07.689235 13590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:07.691440 13590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "10f7e2fc797e43e6b1257833155c4b63"
format_stamp: "Formatted at 2026-08-08 14:29:07 on dist-test-slave-w206"
server_key: "9534408a4c845ed1b23dd5fd16401c67"
server_key_iv: "73a970511114b525ed210be9e02d93d3"
server_key_version: "encryptionkey@0"
I20260808 14:29:07.692022 13590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "10f7e2fc797e43e6b1257833155c4b63"
format_stamp: "Formatted at 2026-08-08 14:29:07 on dist-test-slave-w206"
server_key: "9534408a4c845ed1b23dd5fd16401c67"
server_key_iv: "73a970511114b525ed210be9e02d93d3"
server_key_version: "encryptionkey@0"
I20260808 14:29:07.695888 13590 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:29:07.698515 13606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:07.699873 13590 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 14:29:07.700026 13590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "10f7e2fc797e43e6b1257833155c4b63"
format_stamp: "Formatted at 2026-08-08 14:29:07 on dist-test-slave-w206"
server_key: "9534408a4c845ed1b23dd5fd16401c67"
server_key_iv: "73a970511114b525ed210be9e02d93d3"
server_key_version: "encryptionkey@0"
I20260808 14:29:07.700146 13590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:07.715339 13590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:07.718269 13590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:07.718485 13590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:07.719213 13590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:07.720077 13590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:07.720149 13590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:07.720220 13590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:07.720271 13590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:07.729718 13590 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:44721
I20260808 14:29:07.729737 13726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:44721 every 8 connection(s)
I20260808 14:29:07.730660 13590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:29:07.735098 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 13590
I20260808 14:29:07.735251 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:29:07.735558 7669 external_mini_cluster.cc:1428] Setting key bf1e6aa066ae74fb9817ffd73c6a364d
Aug 08 14:29:07 dist-test-slave-w206 krb5kdc[13476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199347, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:29:07.745780 13567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:07.732578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:50797 (local address 127.7.125.126:42711)
0808 14:29:07.732727 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:07.732731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:07.733563 (+ 832us) server_negotiation.cc:407] Connection header received
0808 14:29:07.735068 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:07.735071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:07.735131 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:07.735239 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:07.736477 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:07.737117 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:07.738804 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:07.739101 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:07.739220 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:07.741835 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:07.741838 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:07.741840 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:07.741865 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:07.743431 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:07.744088 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:07.744090 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:07.744092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:07.744169 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:07.744578 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:07.744580 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:07.744581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:07.744758 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:07.744841 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:07.745531 (+ 690us) server_negotiation.cc:299] Negotiation successful
0808 14:29:07.745650 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
WARNING: no policy specified for kudu/127.7.125.66@KRBTEST.COM; defaulting to no policy
I20260808 14:29:07.747088 13729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:07.732945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:42711 (local address 127.7.125.65:50797)
0808 14:29:07.733411 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:07.733454 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:07.734823 (+ 1369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:07.735421 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:07.735427 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:07.735834 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:07.736326 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:07.736345 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:07.737243 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:07.737247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:07.738665 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:07.738679 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:07.739268 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:07.739273 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:07.739470 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:07.740139 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:07.740155 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:07.741652 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:07.743556 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:07.743565 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:07.743586 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:07.743984 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:07.744275 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:07.744278 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:07.744280 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:07.744468 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:07.744871 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:07.744879 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:07.745245 (+ 366us) client_negotiation.cc:769] Sending connection context
0808 14:29:07.745484 (+ 239us) client_negotiation.cc:240] Negotiation successful
0808 14:29:07.745877 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":134,"threads_started":1}
I20260808 14:29:07.748318 13727 heartbeater.cc:347] Connected to a master server at 127.7.125.126:42711
I20260808 14:29:07.748605 13727 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:07.749169 13727 heartbeater.cc:511] Master 127.7.125.126:42711 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.66@KRBTEST.COM" created.
I20260808 14:29:07.750783 13524 ts_manager.cc:194] Registered new tserver with Master: 10f7e2fc797e43e6b1257833155c4b63 (127.7.125.65:44721)
I20260808 14:29:07.752002 13524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:50797
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
I20260808 14:29:07.766229 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:42711
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:07.880632 13732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:07.880916 13732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:07.880995 13732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:07.885247 13732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:07.885308 13732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:07.885416 13732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:29:07.890686 13732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:42711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:07.891785 13732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:07.892884 13732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:07.902580 13738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:07.902657 13737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:07.902691 13740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:07.902815 13732 server_base.cc:1034] running on GCE node
I20260808 14:29:07.903380 13732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:07.904204 13732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:07.905427 13732 hybrid_clock.cc:648] HybridClock initialized: now 1786199347905396 us; error 42 us; skew 500 ppm
Aug 08 14:29:07 dist-test-slave-w206 krb5kdc[13476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199347, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:07.908649 13732 init.cc:377] Logged in from keytab as kudu/127.7.125.66@KRBTEST.COM (short username kudu)
I20260808 14:29:07.909556 13732 webserver.cc:460] Webserver started at http://127.7.125.66:45345/ using document root <none> and password file <none>
I20260808 14:29:07.910106 13732 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:07.910180 13732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:07.910395 13732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:07.912637 13732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "11fd62ce8cf3402fb8a32e1c4b722d1a"
format_stamp: "Formatted at 2026-08-08 14:29:07 on dist-test-slave-w206"
server_key: "bc71f6c45a3fe24a776d6e79afe5e2bc"
server_key_iv: "57508f588573584ccf8f6ca69d07afbf"
server_key_version: "encryptionkey@0"
I20260808 14:29:07.913226 13732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "11fd62ce8cf3402fb8a32e1c4b722d1a"
format_stamp: "Formatted at 2026-08-08 14:29:07 on dist-test-slave-w206"
server_key: "bc71f6c45a3fe24a776d6e79afe5e2bc"
server_key_iv: "57508f588573584ccf8f6ca69d07afbf"
server_key_version: "encryptionkey@0"
I20260808 14:29:07.916992 13732 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:29:07.919350 13747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:07.920583 13732 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:29:07.920720 13732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "11fd62ce8cf3402fb8a32e1c4b722d1a"
format_stamp: "Formatted at 2026-08-08 14:29:07 on dist-test-slave-w206"
server_key: "bc71f6c45a3fe24a776d6e79afe5e2bc"
server_key_iv: "57508f588573584ccf8f6ca69d07afbf"
server_key_version: "encryptionkey@0"
I20260808 14:29:07.920835 13732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:07.939431 13732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:07.943042 13732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:07.943296 13732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:07.943936 13732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:07.944788 13732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:07.944861 13732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:07.944931 13732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:07.944974 13732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:07.953994 13732 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:40537
I20260808 14:29:07.954017 13867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:40537 every 8 connection(s)
I20260808 14:29:07.954905 13732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:29:07.963917 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 13732
I20260808 14:29:07.964087 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:29:07.964413 7669 external_mini_cluster.cc:1428] Setting key 965bdcee7015c8605d47445385cfc896
Aug 08 14:29:07 dist-test-slave-w206 krb5kdc[13476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199347, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:29:07.971276 13567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:07.956772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:33975 (local address 127.7.125.126:42711)
0808 14:29:07.956954 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:07.956959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:07.957844 (+ 885us) server_negotiation.cc:407] Connection header received
0808 14:29:07.959323 (+ 1479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:07.959326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:07.959374 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:07.959459 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:07.960726 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:07.961318 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:07.963029 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:07.963329 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:07.963425 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:07.966696 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:07.966701 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:07.966703 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:07.966733 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:07.968356 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:07.969238 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:07.969244 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:07.969246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:07.969362 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:07.969837 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:07.969842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:07.969844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:07.970134 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:07.970247 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:07.970847 (+ 600us) server_negotiation.cc:299] Negotiation successful
0808 14:29:07.971022 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 14:29:07.972133 13870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:07.957171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:42711 (local address 127.7.125.66:33975)
0808 14:29:07.957674 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:07.957728 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:07.959129 (+ 1401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:07.959622 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:07.959629 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:07.960042 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:07.960523 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:07.960537 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:07.961450 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:07.961453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:07.962846 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:07.962865 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:07.963556 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:07.963564 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:07.963847 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:07.964545 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:07.964567 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:07.966486 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:07.968494 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:07.968505 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:07.968526 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:07.969058 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:07.969519 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:07.969521 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:07.969523 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:07.969709 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:07.970285 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:07.970291 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:07.970561 (+ 270us) client_negotiation.cc:769] Sending connection context
0808 14:29:07.970739 (+ 178us) client_negotiation.cc:240] Negotiation successful
0808 14:29:07.971146 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":145,"threads_started":1}
I20260808 14:29:07.973428 13868 heartbeater.cc:347] Connected to a master server at 127.7.125.126:42711
I20260808 14:29:07.973723 13868 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:07.974411 13868 heartbeater.cc:511] Master 127.7.125.126:42711 requested a full tablet report, sending...
I20260808 14:29:07.975742 13524 ts_manager.cc:194] Registered new tserver with Master: 11fd62ce8cf3402fb8a32e1c4b722d1a (127.7.125.66:40537)
I20260808 14:29:07.976349 13524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:33975
WARNING: no policy specified for kudu/127.7.125.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
I20260808 14:29:07.997973 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:42711
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:08.112059 13873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:08.112360 13873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:08.112411 13873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:08.116408 13873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:08.116465 13873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:08.116523 13873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:29:08.121654 13873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:42711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:08.122689 13873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:08.123817 13873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:08.133695 13881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:08.133697 13878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:08.133749 13879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:08.133796 13873 server_base.cc:1034] running on GCE node
I20260808 14:29:08.134385 13873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:08.135061 13873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:08.136233 13873 hybrid_clock.cc:648] HybridClock initialized: now 1786199348136213 us; error 30 us; skew 500 ppm
Aug 08 14:29:08 dist-test-slave-w206 krb5kdc[13476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199348, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:08.139138 13873 init.cc:377] Logged in from keytab as kudu/127.7.125.67@KRBTEST.COM (short username kudu)
I20260808 14:29:08.140041 13873 webserver.cc:460] Webserver started at http://127.7.125.67:43067/ using document root <none> and password file <none>
I20260808 14:29:08.140856 13873 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:08.140970 13873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:08.141217 13873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:08.143371 13873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "bb7de42c5f324132a27c62d51ce81d3b"
format_stamp: "Formatted at 2026-08-08 14:29:08 on dist-test-slave-w206"
server_key: "1e337e3f1ac49cc77f97c79221168031"
server_key_iv: "9ff52601082ba5dae1d6c807f23d72ee"
server_key_version: "encryptionkey@0"
I20260808 14:29:08.143931 13873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "bb7de42c5f324132a27c62d51ce81d3b"
format_stamp: "Formatted at 2026-08-08 14:29:08 on dist-test-slave-w206"
server_key: "1e337e3f1ac49cc77f97c79221168031"
server_key_iv: "9ff52601082ba5dae1d6c807f23d72ee"
server_key_version: "encryptionkey@0"
I20260808 14:29:08.147727 13873 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:29:08.150063 13889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:08.151404 13873 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:29:08.151544 13873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "bb7de42c5f324132a27c62d51ce81d3b"
format_stamp: "Formatted at 2026-08-08 14:29:08 on dist-test-slave-w206"
server_key: "1e337e3f1ac49cc77f97c79221168031"
server_key_iv: "9ff52601082ba5dae1d6c807f23d72ee"
server_key_version: "encryptionkey@0"
I20260808 14:29:08.151657 13873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:08.192687 13873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:08.195926 13873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:08.196152 13873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:08.196794 13873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:08.197767 13873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:08.197862 13873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:08.197989 13873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:08.198035 13873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:08.207397 13873 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:46601
I20260808 14:29:08.207417 14012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:46601 every 8 connection(s)
I20260808 14:29:08.208468 13873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:29:08.215976 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 13873
I20260808 14:29:08.216123 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:29:08.216437 7669 external_mini_cluster.cc:1428] Setting key 3419541530eeb6ed55bdedb80b3caa1b
Aug 08 14:29:08 dist-test-slave-w206 krb5kdc[13476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199348, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:29:08.223672 13567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:08.210687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:38221 (local address 127.7.125.126:42711)
0808 14:29:08.210849 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:08.210853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:08.211969 (+ 1116us) server_negotiation.cc:407] Connection header received
0808 14:29:08.213473 (+ 1504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:08.213476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:08.213527 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:08.213612 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:08.214880 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:08.215479 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:08.216759 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:08.217001 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:08.217089 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:08.220066 (+ 2977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:08.220069 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:08.220072 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:08.220101 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:08.221484 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:08.222019 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:08.222023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:08.222025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:08.222112 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:08.222465 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:08.222469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:08.222471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:08.222662 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:08.222771 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:08.223353 (+ 582us) server_negotiation.cc:299] Negotiation successful
0808 14:29:08.223498 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 14:29:08.224556 14015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:08.211203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:42711 (local address 127.7.125.67:38221)
0808 14:29:08.211811 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:08.211858 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:08.213192 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:08.213813 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:08.213823 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:08.214222 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:08.214719 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:08.214732 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:08.215597 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:08.215599 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:08.216637 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:08.216646 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:08.217157 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:08.217161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:08.217406 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:08.218094 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:08.218115 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:08.219885 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:08.221607 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:08.221612 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:08.221628 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:08.221910 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:08.222207 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:08.222210 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:08.222212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:08.222366 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:08.222770 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:08.222779 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:08.223107 (+ 328us) client_negotiation.cc:769] Sending connection context
0808 14:29:08.223299 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 14:29:08.223644 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":149,"threads_started":1}
I20260808 14:29:08.225695 14013 heartbeater.cc:347] Connected to a master server at 127.7.125.126:42711
I20260808 14:29:08.225947 14013 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:08.226465 14013 heartbeater.cc:511] Master 127.7.125.126:42711 requested a full tablet report, sending...
I20260808 14:29:08.227557 13524 ts_manager.cc:194] Registered new tserver with Master: bb7de42c5f324132a27c62d51ce81d3b (127.7.125.67:46601)
I20260808 14:29:08.228120 13524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:38221
I20260808 14:29:08.230614 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/test-admin.keytab.
Aug 08 14:29:08 dist-test-slave-w206 krb5kdc[13476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:08.253046 7669 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 14:29:08.256227 14020 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:08.254620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39164 (local address 127.7.125.65:44721)
0808 14:29:08.254953 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:08.254959 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:08.255068 (+ 109us) server_negotiation.cc:407] Connection header received
0808 14:29:08.255161 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:08.255169 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:08.255417 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:08.255545 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:08.256053 (+ 508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39164: BlockingRecv error: recv got EOF from 127.0.0.1:39164 (error 108)
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":109,"threads_started":1}
W20260808 14:29:08.256333 7669 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/krb5cc' of type FILE
Aug 08 14:29:08 dist-test-slave-w206 krb5kdc[13476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:08.257772 7669 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 14:29:08 dist-test-slave-w206 krb5kdc[13476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:08.258765 7669 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 14:29:08 dist-test-slave-w206 krb5kdc[13476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.65@KRBTEST.COM
I20260808 14:29:08.268869 14020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:08.259709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39174 (local address 127.7.125.65:44721)
0808 14:29:08.259863 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:08.259868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:08.259979 (+ 111us) server_negotiation.cc:407] Connection header received
0808 14:29:08.260077 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:08.260079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:08.260116 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:08.260184 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:08.260689 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:08.261587 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:08.262542 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:08.262913 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:08.263066 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:08.265434 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:08.265443 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:08.265456 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:08.265478 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:08.267282 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:08.267690 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:08.267695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:08.267696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:08.267773 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:08.268035 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:08.268038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:08.268039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:08.268289 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:08.268455 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:08.268618 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 14:29:08.268742 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260808 14:29:08.272709 14020 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:08.271793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39184 (local address 127.7.125.65:44721)
0808 14:29:08.271936 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:08.271940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:08.271968 (+ 28us) server_negotiation.cc:407] Connection header received
0808 14:29:08.272082 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:08.272084 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:08.272122 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:08.272187 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:08.272595 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39184: BlockingRecv error: recv got EOF from 127.0.0.1:39184 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260808 14:29:08.272989 7669 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/krb5cc)
Aug 08 14:29:08 dist-test-slave-w206 krb5kdc[13476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:08.274036 7669 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 14:29:08 dist-test-slave-w206 krb5kdc[13476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:08.275070 7669 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 14:29:08 dist-test-slave-w206 krb5kdc[13476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.65@KRBTEST.COM
I20260808 14:29:08.283403 14020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:08.275879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39194 (local address 127.7.125.65:44721)
0808 14:29:08.276022 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:08.276026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:08.276091 (+ 65us) server_negotiation.cc:407] Connection header received
0808 14:29:08.276177 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:08.276179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:08.276213 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:08.276283 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:08.276746 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:08.277304 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:08.278036 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:08.278289 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:08.278374 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:08.280681 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:08.280684 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:08.280686 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:08.280707 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:08.281910 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:08.282276 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:08.282278 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:08.282280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:08.282350 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:08.282576 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:08.282578 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:08.282579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:08.282728 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:08.282841 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:08.283082 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 14:29:08.283212 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 14:29:08.286094 14020 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:08.285141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39202 (local address 127.7.125.65:44721)
0808 14:29:08.285271 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:08.285275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:08.285308 (+ 33us) server_negotiation.cc:407] Connection header received
0808 14:29:08.285424 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:08.285426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:08.285458 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:08.285520 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:08.286012 (+ 492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39202: BlockingRecv error: recv got EOF from 127.0.0.1:39202 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 14:29:08.286202 7669 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199316863421-7669-0/krb5kdc/krb5cc' of type FILE
Aug 08 14:29:08 dist-test-slave-w206 krb5kdc[13476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:08.287300 7669 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 14:29:08 dist-test-slave-w206 krb5kdc[13476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:08.288163 7669 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 14:29:08 dist-test-slave-w206 krb5kdc[13476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.65@KRBTEST.COM
I20260808 14:29:08.296526 14020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:08.288908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39218 (local address 127.7.125.65:44721)
0808 14:29:08.289038 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:08.289042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:08.289173 (+ 131us) server_negotiation.cc:407] Connection header received
0808 14:29:08.289311 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:08.289316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:08.289359 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:08.289443 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:08.289925 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:08.290420 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:08.291253 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:08.291474 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:08.291583 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:08.293684 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:08.293693 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:08.293695 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:08.293716 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:08.294909 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:08.295348 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:08.295352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:08.295354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:08.295439 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:08.295719 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:08.295722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:08.295724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:08.295882 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:08.296043 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:08.296262 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 14:29:08.296370 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 14:29:08.297552 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 13590
I20260808 14:29:08.305838 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 13732
I20260808 14:29:08.316356 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 13873
I20260808 14:29:08.323462 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 13492
2026-08-08T14:29:08Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2316 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:29:08 dist-test-slave-w206 krb5kdc[14034](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:29:08 dist-test-slave-w206 krb5kdc[14034](info): set up 2 sockets
Aug 08 14:29:08 dist-test-slave-w206 krb5kdc[14034](info): commencing operation
krb5kdc: starting...
W20260808 14:29:09.407919 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:29:09 dist-test-slave-w206 krb5kdc[14034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:29:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:29:09Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.126.keytab.
Entry for principal oryx/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.126.keytab.
I20260808 14:29:09.538367 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:37133
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.126.keytab
--principal=oryx/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:37133
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:09.649761 14048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:09.650009 14048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:09.650049 14048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:09.654074 14048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:29:09.654130 14048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:09.654168 14048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:09.654223 14048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:29:09.654258 14048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:29:09.659423 14048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:37133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.126.keytab
--principal=oryx/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:37133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:09.660457 14048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:09.661474 14048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:09.669979 14053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:09.670020 14048 server_base.cc:1034] running on GCE node
W20260808 14:29:09.669973 14056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:09.669973 14054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:09.670871 14048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:09.671923 14048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:09.673089 14048 hybrid_clock.cc:648] HybridClock initialized: now 1786199349673064 us; error 38 us; skew 500 ppm
Aug 08 14:29:09 dist-test-slave-w206 krb5kdc[14034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199349, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:09.675908 14048 init.cc:377] Logged in from keytab as oryx/127.7.125.126@KRBTEST.COM (short username oryx)
I20260808 14:29:09.676774 14048 webserver.cc:460] Webserver started at http://127.7.125.126:36359/ using document root <none> and password file <none>
I20260808 14:29:09.677330 14048 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:09.677443 14048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:09.677672 14048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:09.679736 14048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "d1fbb4311136492580021310c6a777e6"
format_stamp: "Formatted at 2026-08-08 14:29:09 on dist-test-slave-w206"
server_key: "a62ad84b97c8d8c864b466b2623a69ea"
server_key_iv: "bd57d60a38c8b45077619ea04348a58a"
server_key_version: "encryptionkey@0"
I20260808 14:29:09.680295 14048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "d1fbb4311136492580021310c6a777e6"
format_stamp: "Formatted at 2026-08-08 14:29:09 on dist-test-slave-w206"
server_key: "a62ad84b97c8d8c864b466b2623a69ea"
server_key_iv: "bd57d60a38c8b45077619ea04348a58a"
server_key_version: "encryptionkey@0"
I20260808 14:29:09.683959 14048 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:29:09.686299 14064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:09.687542 14048 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:29:09.687680 14048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "d1fbb4311136492580021310c6a777e6"
format_stamp: "Formatted at 2026-08-08 14:29:09 on dist-test-slave-w206"
server_key: "a62ad84b97c8d8c864b466b2623a69ea"
server_key_iv: "bd57d60a38c8b45077619ea04348a58a"
server_key_version: "encryptionkey@0"
I20260808 14:29:09.687785 14048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:09.704821 14048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:09.707808 14048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:09.708007 14048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:09.714808 14048 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:37133
I20260808 14:29:09.715116 14119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:37133 every 8 connection(s)
I20260808 14:29:09.715848 14048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:29:09.718521 14120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:09.723817 14120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6: Bootstrap starting.
I20260808 14:29:09.725072 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 14048
I20260808 14:29:09.725215 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:29:09.725494 7669 external_mini_cluster.cc:1428] Setting key 8c00f261bde2f2e24e9e4c98481043c0
I20260808 14:29:09.726398 14120 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:09.727211 14120 log.cc:824] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:09.729260 14120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6: No bootstrap required, opened a new log
I20260808 14:29:09.733529 14120 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } }
I20260808 14:29:09.733819 14120 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:09.733907 14120 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1fbb4311136492580021310c6a777e6, State: Initialized, Role: FOLLOWER
Aug 08 14:29:09 dist-test-slave-w206 krb5kdc[14034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM
I20260808 14:29:09.734474 14120 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } }
I20260808 14:29:09.734624 14120 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:29:09.734730 14120 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:29:09.734855 14120 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:09.735788 14120 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } }
I20260808 14:29:09.736125 14120 leader_election.cc:302] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1fbb4311136492580021310c6a777e6; no voters:
I20260808 14:29:09.736433 14120 leader_election.cc:288] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:29:09.736533 14125 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:29:09.736740 14125 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [term 1 LEADER]: Becoming Leader. State: Replica: d1fbb4311136492580021310c6a777e6, State: Running, Role: LEADER
I20260808 14:29:09.737128 14125 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } }
I20260808 14:29:09.737778 14120 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:29:09.738739 14126 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1fbb4311136492580021310c6a777e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } }
I20260808 14:29:09.738857 14126 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:09.739120 14126 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1fbb4311136492580021310c6a777e6. Latest consensus state: current_term: 1 leader_uuid: "d1fbb4311136492580021310c6a777e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } }
I20260808 14:29:09.739210 14126 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:09.740310 14134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:29:09.741297 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:09.726575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55942 (local address 127.7.125.126:37133)
0808 14:29:09.727092 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:09.727108 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:09.727142 (+ 34us) server_negotiation.cc:407] Connection header received
0808 14:29:09.727814 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:09.727831 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:09.728138 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:09.728409 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:09.729297 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:09.730312 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:09.731536 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:09.732008 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:09.732123 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:09.735225 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:09.735230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:09.735243 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:09.735269 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:09.737431 (+ 2162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:09.737972 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:09.737978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:09.737983 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:09.738135 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:09.738525 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:09.738529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:09.738532 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:09.739128 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:09.740036 (+ 908us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:09.740243 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 14:29:09.740565 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":184,"threads_started":1}
I20260808 14:29:09.743342 14134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:29:09.748486 14134 catalog_manager.cc:1347] Generated new cluster ID: e6e375437ebd404ab1a6a3c7859fe228
I20260808 14:29:09.748543 14134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:29:09.755579 14134 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:29:09.756418 14134 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:29:09.760229 14134 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6: Generated new TSK 0
I20260808 14:29:09.760814 14134 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.125.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.125.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.125.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.65.keytab.
Entry for principal oryx/127.7.125.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.65.keytab.
I20260808 14:29:09.794843 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:37133
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.65.keytab
--principal=oryx/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:09.924368 14147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:09.924633 14147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:09.924721 14147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:09.928797 14147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:09.928867 14147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:09.928952 14147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:29:09.934027 14147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.65.keytab
--principal=oryx/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:37133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:09.935081 14147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:09.936071 14147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:09.945676 14153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:09.945709 14152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:09.945844 14147 server_base.cc:1034] running on GCE node
W20260808 14:29:09.945729 14155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:09.946301 14147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:09.946928 14147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:09.948127 14147 hybrid_clock.cc:648] HybridClock initialized: now 1786199349948107 us; error 30 us; skew 500 ppm
Aug 08 14:29:09 dist-test-slave-w206 krb5kdc[14034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199349, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:09.950760 14147 init.cc:377] Logged in from keytab as oryx/127.7.125.65@KRBTEST.COM (short username oryx)
I20260808 14:29:09.951574 14147 webserver.cc:460] Webserver started at http://127.7.125.65:38209/ using document root <none> and password file <none>
I20260808 14:29:09.952066 14147 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:09.952124 14147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:09.952325 14147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:09.954313 14147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "77f51c177c5f49679dfb01ce597d07c8"
format_stamp: "Formatted at 2026-08-08 14:29:09 on dist-test-slave-w206"
server_key: "91027b6978d441bc479f09824bd40e5b"
server_key_iv: "8567d249351156327ebb25a1249c3074"
server_key_version: "encryptionkey@0"
I20260808 14:29:09.954854 14147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "77f51c177c5f49679dfb01ce597d07c8"
format_stamp: "Formatted at 2026-08-08 14:29:09 on dist-test-slave-w206"
server_key: "91027b6978d441bc479f09824bd40e5b"
server_key_iv: "8567d249351156327ebb25a1249c3074"
server_key_version: "encryptionkey@0"
I20260808 14:29:09.958349 14147 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260808 14:29:09.960948 14162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:09.962108 14147 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:29:09.962245 14147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "77f51c177c5f49679dfb01ce597d07c8"
format_stamp: "Formatted at 2026-08-08 14:29:09 on dist-test-slave-w206"
server_key: "91027b6978d441bc479f09824bd40e5b"
server_key_iv: "8567d249351156327ebb25a1249c3074"
server_key_version: "encryptionkey@0"
I20260808 14:29:09.962352 14147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:09.974321 14147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:09.977118 14147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:09.977317 14147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:09.977883 14147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:09.978694 14147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:09.978763 14147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:09.978838 14147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:09.978884 14147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:09.987998 14147 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:35815
I20260808 14:29:09.988027 14283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:35815 every 8 connection(s)
I20260808 14:29:09.988917 14147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:29:09.992997 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 14147
I20260808 14:29:09.993147 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:29:09.993490 7669 external_mini_cluster.cc:1428] Setting key bb28514352fe6b966db523a861fe2471
Aug 08 14:29:09 dist-test-slave-w206 krb5kdc[14034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199349, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM
I20260808 14:29:10.004262 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:09.990711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:56799 (local address 127.7.125.126:37133)
0808 14:29:09.990869 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:09.990873 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:09.992067 (+ 1194us) server_negotiation.cc:407] Connection header received
0808 14:29:09.992995 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:09.992998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:09.993044 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:09.993143 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:09.994608 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:09.995303 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:09.997071 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:09.997492 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:09.997607 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:10.000131 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:10.000137 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:10.000142 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:10.000178 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:10.001973 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:10.002491 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:10.002496 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:10.002501 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:10.002603 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:10.002951 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:10.002956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:10.002958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:10.003254 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:10.003379 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:10.003856 (+ 477us) server_negotiation.cc:299] Negotiation successful
0808 14:29:10.004022 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
WARNING: no policy specified for oryx/127.7.125.66@KRBTEST.COM; defaulting to no policy
I20260808 14:29:10.005035 14286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:09.991164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.65:56799)
0808 14:29:09.991906 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:09.991963 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:09.992748 (+ 785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:09.993350 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:09.993360 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:09.993884 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:09.994455 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:09.994470 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:09.995439 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:09.995445 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:09.996917 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:09.996932 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:09.997653 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:09.997656 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:09.997794 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:09.998405 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:09.998422 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:09.999925 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:10.002095 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:10.002100 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:10.002113 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:10.002384 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:10.002703 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:10.002705 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:10.002707 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:10.002853 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:10.003360 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:10.003366 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:10.003605 (+ 239us) client_negotiation.cc:769] Sending connection context
0808 14:29:10.003764 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 14:29:10.004045 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":135,"threads_started":1}
I20260808 14:29:10.006319 14284 heartbeater.cc:347] Connected to a master server at 127.7.125.126:37133
I20260808 14:29:10.006575 14284 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:10.007153 14284 heartbeater.cc:511] Master 127.7.125.126:37133 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.125.66@KRBTEST.COM" created.
I20260808 14:29:10.008845 14082 ts_manager.cc:194] Registered new tserver with Master: 77f51c177c5f49679dfb01ce597d07c8 (127.7.125.65:35815)
I20260808 14:29:10.010417 14082 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:56799
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.125.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.66.keytab.
Entry for principal oryx/127.7.125.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.66.keytab.
I20260808 14:29:10.026340 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:37133
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.66.keytab
--principal=oryx/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:10.139964 14289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:10.140230 14289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:10.140302 14289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:10.144340 14289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:10.144398 14289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:10.144495 14289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:29:10.149667 14289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.66.keytab
--principal=oryx/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:37133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:10.150722 14289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:10.151777 14289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:10.161679 14296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:10.161679 14294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:10.161718 14298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:10.162062 14289 server_base.cc:1034] running on GCE node
I20260808 14:29:10.162547 14289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:10.163285 14289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:10.164489 14289 hybrid_clock.cc:648] HybridClock initialized: now 1786199350164454 us; error 51 us; skew 500 ppm
Aug 08 14:29:10 dist-test-slave-w206 krb5kdc[14034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199350, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:10.167562 14289 init.cc:377] Logged in from keytab as oryx/127.7.125.66@KRBTEST.COM (short username oryx)
I20260808 14:29:10.168576 14289 webserver.cc:460] Webserver started at http://127.7.125.66:43607/ using document root <none> and password file <none>
I20260808 14:29:10.169153 14289 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:10.169242 14289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:10.169459 14289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:10.171607 14289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "6907de3a9a4144acbce31a6b7f0d6200"
format_stamp: "Formatted at 2026-08-08 14:29:10 on dist-test-slave-w206"
server_key: "d6d37bb3a022013dd9a9d92fe14be22a"
server_key_iv: "0e74222b893b6768982033fe4df76e7d"
server_key_version: "encryptionkey@0"
I20260808 14:29:10.172170 14289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "6907de3a9a4144acbce31a6b7f0d6200"
format_stamp: "Formatted at 2026-08-08 14:29:10 on dist-test-slave-w206"
server_key: "d6d37bb3a022013dd9a9d92fe14be22a"
server_key_iv: "0e74222b893b6768982033fe4df76e7d"
server_key_version: "encryptionkey@0"
I20260808 14:29:10.175864 14289 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:29:10.178165 14306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:10.179388 14289 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:29:10.179515 14289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "6907de3a9a4144acbce31a6b7f0d6200"
format_stamp: "Formatted at 2026-08-08 14:29:10 on dist-test-slave-w206"
server_key: "d6d37bb3a022013dd9a9d92fe14be22a"
server_key_iv: "0e74222b893b6768982033fe4df76e7d"
server_key_version: "encryptionkey@0"
I20260808 14:29:10.179620 14289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:10.201692 14289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:10.204663 14289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:10.204869 14289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:10.205493 14289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:10.206310 14289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:10.206385 14289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:10.206454 14289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:10.206494 14289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:10.215790 14289 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:44269
I20260808 14:29:10.215824 14422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:44269 every 8 connection(s)
I20260808 14:29:10.216701 14289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:29:10.223574 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 14289
I20260808 14:29:10.223702 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:29:10.224030 7669 external_mini_cluster.cc:1428] Setting key fcf951998a082b17f383f305cb61c800
Aug 08 14:29:10 dist-test-slave-w206 krb5kdc[14034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199350, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM
I20260808 14:29:10.230674 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:10.218616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:34961 (local address 127.7.125.126:37133)
0808 14:29:10.218796 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:10.218801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:10.219959 (+ 1158us) server_negotiation.cc:407] Connection header received
0808 14:29:10.220872 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:10.220875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:10.220924 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:10.221001 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:10.222188 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.222724 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.224198 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.224455 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.224562 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:10.227148 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:10.227151 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:10.227154 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:10.227177 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:10.228672 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:10.229204 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:10.229208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:10.229209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:10.229282 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:10.229617 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:10.229621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:10.229622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:10.229797 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:10.229905 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:10.230382 (+ 477us) server_negotiation.cc:299] Negotiation successful
0808 14:29:10.230509 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 14:29:10.231771 14425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:10.219127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.66:34961)
0808 14:29:10.219798 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:10.219841 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:10.220667 (+ 826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:10.221163 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:10.221170 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:10.221572 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:10.222057 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:10.222071 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.222838 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.222841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:10.224074 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:10.224086 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.224634 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.224640 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:10.224846 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:10.225461 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:10.225484 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:10.226941 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:10.228789 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:10.228795 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:10.228810 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:10.229080 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:10.229372 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:10.229376 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:10.229379 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:10.229520 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:10.229903 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:10.229910 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:10.230149 (+ 239us) client_negotiation.cc:769] Sending connection context
0808 14:29:10.230331 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 14:29:10.230642 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":112,"threads_started":1}
WARNING: no policy specified for oryx/127.7.125.67@KRBTEST.COM; defaulting to no policy
I20260808 14:29:10.235939 14423 heartbeater.cc:347] Connected to a master server at 127.7.125.126:37133
I20260808 14:29:10.236218 14423 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:10.236908 14423 heartbeater.cc:511] Master 127.7.125.126:37133 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.125.67@KRBTEST.COM" created.
I20260808 14:29:10.238237 14082 ts_manager.cc:194] Registered new tserver with Master: 6907de3a9a4144acbce31a6b7f0d6200 (127.7.125.66:44269)
I20260808 14:29:10.239190 14082 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:34961
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.125.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.67.keytab.
Entry for principal oryx/127.7.125.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.67.keytab.
I20260808 14:29:10.254449 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:37133
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.67.keytab
--principal=oryx/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:10.368053 14428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:10.368342 14428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:10.368423 14428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:10.372596 14428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:10.372684 14428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:10.372777 14428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:29:10.378149 14428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.67.keytab
--principal=oryx/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:37133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:10.379328 14428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:10.380417 14428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:10.390002 14434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:10.390043 14436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:10.390043 14433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:10.390225 14428 server_base.cc:1034] running on GCE node
I20260808 14:29:10.390717 14428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:10.391433 14428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:10.392585 14428 hybrid_clock.cc:648] HybridClock initialized: now 1786199350392577 us; error 47 us; skew 500 ppm
Aug 08 14:29:10 dist-test-slave-w206 krb5kdc[14034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199350, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:10.395268 14428 init.cc:377] Logged in from keytab as oryx/127.7.125.67@KRBTEST.COM (short username oryx)
I20260808 14:29:10.396132 14428 webserver.cc:460] Webserver started at http://127.7.125.67:45423/ using document root <none> and password file <none>
I20260808 14:29:10.396656 14428 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:10.396714 14428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:10.396878 14428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:10.398849 14428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "fef7bd05dbce458392c26470cccb0bb7"
format_stamp: "Formatted at 2026-08-08 14:29:10 on dist-test-slave-w206"
server_key: "3ce2488e2652450ee27d9b9f791d8788"
server_key_iv: "6c0d30ec675057063cf15286416bf180"
server_key_version: "encryptionkey@0"
I20260808 14:29:10.399447 14428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "fef7bd05dbce458392c26470cccb0bb7"
format_stamp: "Formatted at 2026-08-08 14:29:10 on dist-test-slave-w206"
server_key: "3ce2488e2652450ee27d9b9f791d8788"
server_key_iv: "6c0d30ec675057063cf15286416bf180"
server_key_version: "encryptionkey@0"
I20260808 14:29:10.403120 14428 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:29:10.405469 14446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:10.406701 14428 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:29:10.406829 14428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "fef7bd05dbce458392c26470cccb0bb7"
format_stamp: "Formatted at 2026-08-08 14:29:10 on dist-test-slave-w206"
server_key: "3ce2488e2652450ee27d9b9f791d8788"
server_key_iv: "6c0d30ec675057063cf15286416bf180"
server_key_version: "encryptionkey@0"
I20260808 14:29:10.406939 14428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:10.421233 14428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:10.424249 14428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:10.424482 14428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:10.425064 14428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:10.425885 14428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:10.425954 14428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:10.426017 14428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:10.426059 14428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:10.435686 14428 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:37109
I20260808 14:29:10.435709 14565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:37109 every 8 connection(s)
I20260808 14:29:10.436569 14428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:29:10.441792 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 14428
I20260808 14:29:10.441901 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:29:10.442214 7669 external_mini_cluster.cc:1428] Setting key 16c862a40c786f24c857b1b55337ada2
Aug 08 14:29:10 dist-test-slave-w206 krb5kdc[14034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199350, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM
I20260808 14:29:10.449744 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:10.438421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:33715 (local address 127.7.125.126:37133)
0808 14:29:10.438563 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:10.438566 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:10.439691 (+ 1125us) server_negotiation.cc:407] Connection header received
0808 14:29:10.440540 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:10.440543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:10.440590 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:10.440665 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:10.441784 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.442466 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.443942 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.444254 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.444368 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:10.446742 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:10.446745 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:10.446747 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:10.446769 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:10.448056 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:10.448513 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:10.448516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:10.448517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:10.448591 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:10.448875 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:10.448877 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:10.448878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:10.449039 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:10.449112 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:10.449537 (+ 425us) server_negotiation.cc:299] Negotiation successful
0808 14:29:10.449631 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 14:29:10.450544 14568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:10.438779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.67:33715)
0808 14:29:10.439556 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:10.439598 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:10.440359 (+ 761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:10.440820 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:10.440826 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:10.441177 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:10.441632 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:10.441645 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.442593 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.442597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:10.443809 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:10.443818 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.444408 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.444413 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:10.444590 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:10.445104 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:10.445120 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:10.446595 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:10.448157 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:10.448162 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:10.448173 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:10.448431 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:10.448664 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:10.448666 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:10.448668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:10.448800 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:10.449120 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:10.449124 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:10.449336 (+ 212us) client_negotiation.cc:769] Sending connection context
0808 14:29:10.449496 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 14:29:10.449730 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":178,"threads_started":1}
I20260808 14:29:10.451704 14566 heartbeater.cc:347] Connected to a master server at 127.7.125.126:37133
I20260808 14:29:10.451906 14566 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:10.452349 14566 heartbeater.cc:511] Master 127.7.125.126:37133 requested a full tablet report, sending...
I20260808 14:29:10.453276 14082 ts_manager.cc:194] Registered new tserver with Master: fef7bd05dbce458392c26470cccb0bb7 (127.7.125.67:37109)
I20260808 14:29:10.453778 14082 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:33715
I20260808 14:29:10.455999 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.125.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.125.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.125.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.125.keytab.
Entry for principal oryx/127.7.125.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.125.keytab.
I20260808 14:29:10.485685 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.125:33639
--webserver_interface=127.7.125.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.125.keytab
--principal=oryx/127.7.125.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:37133,127.7.125.125:33639
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:10.597462 14571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:10.597718 14571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:10.597759 14571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:10.601847 14571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:29:10.601907 14571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:10.601925 14571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:10.601943 14571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:29:10.601958 14571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:29:10.607141 14571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:37133,127.7.125.125:33639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.125.keytab
--principal=oryx/127.7.125.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.125:33639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.125.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
W20260808 14:29:10.607323 14571 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.7.125.126:37133,127.7.125.125:33639'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 14:29:10.608151 14571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:10.609107 14571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:10.617697 14579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:10.617697 14577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:10.617928 14576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:10.617954 14571 server_base.cc:1034] running on GCE node
I20260808 14:29:10.618575 14571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:10.619696 14571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:10.620857 14571 hybrid_clock.cc:648] HybridClock initialized: now 1786199350620842 us; error 28 us; skew 500 ppm
Aug 08 14:29:10 dist-test-slave-w206 krb5kdc[14034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199350, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:10.623662 14571 init.cc:377] Logged in from keytab as oryx/127.7.125.125@KRBTEST.COM (short username oryx)
I20260808 14:29:10.624516 14571 webserver.cc:460] Webserver started at http://127.7.125.125:43087/ using document root <none> and password file <none>
I20260808 14:29:10.625011 14571 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:10.625068 14571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:10.625231 14571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:10.627271 14571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/data/instance:
uuid: "d87c0c9ba2a540778055acacfc93fc35"
format_stamp: "Formatted at 2026-08-08 14:29:10 on dist-test-slave-w206"
server_key: "f3f573f4915fbf50bd0f1b3d77cdaf23"
server_key_iv: "f0a025bd63f3e63377c992e611ee1503"
server_key_version: "encryptionkey@0"
I20260808 14:29:10.627792 14571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/wal/instance:
uuid: "d87c0c9ba2a540778055acacfc93fc35"
format_stamp: "Formatted at 2026-08-08 14:29:10 on dist-test-slave-w206"
server_key: "f3f573f4915fbf50bd0f1b3d77cdaf23"
server_key_iv: "f0a025bd63f3e63377c992e611ee1503"
server_key_version: "encryptionkey@0"
I20260808 14:29:10.631520 14571 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:29:10.633853 14586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:10.635088 14571 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:29:10.635208 14571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/wal
uuid: "d87c0c9ba2a540778055acacfc93fc35"
format_stamp: "Formatted at 2026-08-08 14:29:10 on dist-test-slave-w206"
server_key: "f3f573f4915fbf50bd0f1b3d77cdaf23"
server_key_iv: "f0a025bd63f3e63377c992e611ee1503"
server_key_version: "encryptionkey@0"
I20260808 14:29:10.635339 14571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:10.653895 14571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:10.656921 14571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:10.657138 14571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:10.663879 14571 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.125:33639
I20260808 14:29:10.664162 14641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.125:33639 every 8 connection(s)
I20260808 14:29:10.664871 14571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/data/info.pb
I20260808 14:29:10.667832 14642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:10.671547 14642 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:29:10.672556 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 14571
Aug 08 14:29:10 dist-test-slave-w206 krb5kdc[14034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199350, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.125@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM
Aug 08 14:29:10 dist-test-slave-w206 krb5kdc[14034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.125.125@KRBTEST.COM
I20260808 14:29:10.686299 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:10.673634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55952 (local address 127.7.125.126:37133)
0808 14:29:10.673785 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:10.673792 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:10.674633 (+ 841us) server_negotiation.cc:407] Connection header received
0808 14:29:10.675613 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:10.675616 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:10.675663 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:10.675755 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:10.676866 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.677692 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.679109 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.679444 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.679549 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:10.682206 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:10.682210 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:10.682213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:10.682240 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:10.684072 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:10.684629 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:10.684636 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:10.684638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:10.684730 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:10.685101 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:10.685104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:10.685106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:10.685387 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:10.685507 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:10.685980 (+ 473us) server_negotiation.cc:299] Negotiation successful
0808 14:29:10.686126 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:29:10.687119 14646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:10.673993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.0.0.1:55952)
0808 14:29:10.674492 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:10.674533 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:10.675430 (+ 897us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:10.675839 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:10.675845 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:10.676341 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:10.676718 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:10.676738 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.677823 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.677830 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:10.678995 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:10.679006 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.679576 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.679578 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:10.679728 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:10.680367 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:10.680414 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:10.682060 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:10.684173 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:10.684180 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:10.684187 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:10.684533 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:10.684823 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:10.684828 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:10.684830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:10.684999 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:10.685505 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:10.685510 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:10.685731 (+ 221us) client_negotiation.cc:769] Sending connection context
0808 14:29:10.685913 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 14:29:10.686216 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":48,"spinlock_wait_cycles":2304,"thread_start_us":161,"threads_started":1}
I20260808 14:29:10.687117 14647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:10.673993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49624 (local address 127.7.125.125:33639)
0808 14:29:10.674924 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:10.674932 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:10.674963 (+ 31us) server_negotiation.cc:407] Connection header received
0808 14:29:10.675437 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:10.675460 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:10.675679 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:10.675857 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:10.676732 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.677970 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.679102 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.679665 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.679769 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:10.682723 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:10.682733 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:10.682746 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:10.682770 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:10.684589 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:10.685112 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:10.685117 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:10.685119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:10.685195 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:10.685449 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:10.685454 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:10.685458 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:10.685715 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:10.685853 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:10.686135 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 14:29:10.686421 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":834,"spinlock_wait_cycles":53632,"thread_start_us":161,"threads_started":1}
I20260808 14:29:10.688583 14642 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 14:29:10 dist-test-slave-w206 krb5kdc[14034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199350, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.125@KRBTEST.COM for oryx/127.7.125.125@KRBTEST.COM
I20260808 14:29:10.695915 14646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:10.689066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:33639 (local address 127.0.0.1:49636)
0808 14:29:10.689152 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:10.689167 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:10.689253 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:10.689579 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:10.689582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:10.689704 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:10.689789 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:10.689793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.690460 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.690463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:10.691084 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:10.691093 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.691504 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.691507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:10.691577 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:10.691980 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:10.691991 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:10.693128 (+ 1137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:10.694678 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:10.694680 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:10.694681 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:10.694855 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:10.695155 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:10.695157 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:10.695158 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:10.695228 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:10.695528 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:10.695530 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:10.695596 (+ 66us) client_negotiation.cc:769] Sending connection context
0808 14:29:10.695674 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 14:29:10.695761 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 14:29:10.695943 14647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:10.689308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49636 (local address 127.7.125.125:33639)
0808 14:29:10.689411 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:10.689414 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:10.689432 (+ 18us) server_negotiation.cc:407] Connection header received
0808 14:29:10.689459 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:10.689461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:10.689492 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:10.689553 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:10.689879 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.690360 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.691183 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.691421 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.691491 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:10.693240 (+ 1749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:10.693243 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:10.693244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:10.693264 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:10.694589 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:10.694935 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:10.694938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:10.694939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:10.695080 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:10.695300 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:10.695302 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:10.695303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:10.695453 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:10.695524 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:10.695723 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 14:29:10.695807 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 14:29:10.699209 14642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35: Bootstrap starting.
I20260808 14:29:10.701363 14642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:10.702057 14642 log.cc:824] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:10.704000 14642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35: No bootstrap required, opened a new log
I20260808 14:29:10.706809 14642 raft_consensus.cc:348] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } }
I20260808 14:29:10.707056 14642 raft_consensus.cc:374] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:10.707144 14642 raft_consensus.cc:729] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d87c0c9ba2a540778055acacfc93fc35, State: Initialized, Role: FOLLOWER
I20260808 14:29:10.707561 14642 consensus_queue.cc:260] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } }
I20260808 14:29:10.708055 14649 sys_catalog.cc:455] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } } }
I20260808 14:29:10.708236 14649 sys_catalog.cc:458] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:29:10.708745 14642 sys_catalog.cc:564] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 14:29:10.713517 14660 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:29:10.713601 14660 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:29:10.715792 14571 master_runner.cc:428] Detected that this master d87c0c9ba2a540778055acacfc93fc35 is joining an existing cluster
I20260808 14:29:10.715862 14571 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.125.125:33639
I20260808 14:29:10.728382 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:10.718284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55962 (local address 127.7.125.126:37133)
0808 14:29:10.718417 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:10.718422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:10.718654 (+ 232us) server_negotiation.cc:407] Connection header received
0808 14:29:10.719025 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:10.719029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:10.719088 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:10.719147 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:10.719818 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.720650 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.721872 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.722211 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.722322 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:10.723760 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:10.723766 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:10.723772 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:10.723798 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:10.725613 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:10.726382 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:10.726386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:10.726388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:10.726491 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:10.726799 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:10.726802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:10.726804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:10.727034 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:10.727375 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:10.728102 (+ 727us) server_negotiation.cc:299] Negotiation successful
0808 14:29:10.728226 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 14:29:10.729233 14666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:10.718185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.0.0.1:55962)
0808 14:29:10.718573 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:10.718588 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:10.718722 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:10.719295 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:10.719298 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:10.719551 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:10.719692 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:10.719699 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.720768 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.720772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:10.721746 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:10.721758 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.722341 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.722345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:10.722457 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:10.723061 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:10.723076 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:10.723598 (+ 522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:10.725750 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:10.725753 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:10.725755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:10.726259 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:10.726602 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:10.726604 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:10.726606 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:10.726697 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:10.727493 (+ 796us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:10.727496 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:10.727588 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 14:29:10.728919 (+ 1331us) client_negotiation.cc:240] Negotiation successful
0808 14:29:10.729060 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":93,"threads_started":1}
I20260808 14:29:10.729889 14647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:10.718862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49642 (local address 127.7.125.125:33639)
0808 14:29:10.718990 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:10.718995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:10.722652 (+ 3657us) server_negotiation.cc:407] Connection header received
0808 14:29:10.722796 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:10.722798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:10.722834 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:10.722876 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:10.723495 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.724223 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.725287 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.725618 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.725678 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:10.726816 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:10.726819 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:10.726822 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:10.726862 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:10.728480 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:10.728877 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:10.728880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:10.728881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:10.728943 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:10.729158 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:10.729161 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:10.729162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:10.729324 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:10.729382 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:10.729651 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 14:29:10.729747 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 14:29:10.730279 14667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:10.718582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:33639 (local address 127.0.0.1:49642)
0808 14:29:10.722574 (+ 3992us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:10.722589 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:10.722706 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:10.723014 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:10.723016 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:10.723273 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:10.723388 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:10.723394 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.724331 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.724334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:10.725177 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:10.725185 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.725793 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.725796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:10.725874 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:10.726297 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:10.726318 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:10.726692 (+ 374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:10.728582 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:10.728584 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:10.728586 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:10.728790 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:10.729025 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:10.729027 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:10.729029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:10.729088 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:10.729433 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:10.729436 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:10.729504 (+ 68us) client_negotiation.cc:769] Sending connection context
0808 14:29:10.730024 (+ 520us) client_negotiation.cc:240] Negotiation successful
0808 14:29:10.730137 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3918,"thread_start_us":77,"threads_started":1}
Aug 08 14:29:10 dist-test-slave-w206 krb5kdc[14034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199349, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.126@KRBTEST.COM for oryx/127.7.125.125@KRBTEST.COM
I20260808 14:29:10.743813 14668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:10.733821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:33639 (local address 127.0.0.1:49646)
0808 14:29:10.734594 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:10.734639 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:10.734768 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:10.735065 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:10.735071 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:10.735492 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:10.735613 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:10.735621 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.736259 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.736263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:10.737246 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:10.737257 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.737654 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.737657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:10.737767 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:10.738305 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:10.738319 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:10.740118 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:10.741990 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:10.742000 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:10.742006 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:10.742290 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:10.742616 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:10.742619 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:10.742621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:10.742703 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:10.743163 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:10.743170 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:10.743293 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 14:29:10.743450 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 14:29:10.743634 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":93,"threads_started":1}
I20260808 14:29:10.743829 14647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:10.733862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49646 (local address 127.7.125.125:33639)
0808 14:29:10.733969 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:10.733972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:10.734711 (+ 739us) server_negotiation.cc:407] Connection header received
0808 14:29:10.734880 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:10.734882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:10.734921 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:10.735027 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:10.735717 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.736160 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.737361 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.737554 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.737626 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:10.740268 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:10.740272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:10.740275 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:10.740298 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:10.741865 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:10.742416 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:10.742419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:10.742421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:10.742504 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:10.742797 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:10.742800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:10.742802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:10.743011 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:10.743080 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:10.743532 (+ 452us) server_negotiation.cc:299] Negotiation successful
0808 14:29:10.743656 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 14:29:10.745666 14082 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.125.125:33639: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35"
member_type: NON_VOTER
last_known_addr {
host: "127.7.125.125"
port: 33639
}
attrs {
promote: true
}
}
dest_uuid: "d1fbb4311136492580021310c6a777e6"
cas_config_opid_index: -1
I20260808 14:29:10.746420 14082 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: NON_VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: true } }
I20260808 14:29:10.748494 14670 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER d87c0c9ba2a540778055acacfc93fc35 (127.7.125.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: NON_VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: true } } }
I20260808 14:29:10.749140 14613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:10.750245 14670 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.125.125:33639
I20260808 14:29:10.750319 14613 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 1 FOLLOWER]: Refusing update from remote peer d1fbb4311136492580021310c6a777e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 14:29:10.750640 14671 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d1fbb4311136492580021310c6a777e6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: NON_VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: true } } }
I20260808 14:29:10.750835 14671 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:10.750968 14571 master.cc:537] Master@127.7.125.125:33639 shutting down...
WARNING: no policy specified for oryx/127.7.125.124@KRBTEST.COM; defaulting to no policy
I20260808 14:29:10.751088 14671 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: NON_VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 14:29:10.754436 14068 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 14:29:10.755523 14068 consensus_peers.cc:487] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 -> Peer d87c0c9ba2a540778055acacfc93fc35 (127.7.125.125:33639): Couldn't send request to peer d87c0c9ba2a540778055acacfc93fc35. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.125.124@KRBTEST.COM" created.
I20260808 14:29:10.756661 14571 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:29:10.756930 14571 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:10.757033 14571 tablet_replica.cc:330] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35: stopping tablet replica
I20260808 14:29:10.762243 14571 master.cc:552] Master@127.7.125.125:33639 shutdown complete.
I20260808 14:29:10.762296 14571 master_runner.cc:305] Clearing existing system tablet
I20260808 14:29:10.762537 14571 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:10.762595 14571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:10.763893 14571 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 14:29:10.764688 14676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:10.764916 14571 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:10.765057 14571 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 14:29:10.765113 14571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/wal
uuid: "d87c0c9ba2a540778055acacfc93fc35"
format_stamp: "Formatted at 2026-08-08 14:29:10 on dist-test-slave-w206"
server_key: "f3f573f4915fbf50bd0f1b3d77cdaf23"
server_key_iv: "f0a025bd63f3e63377c992e611ee1503"
server_key_version: "encryptionkey@0"
I20260808 14:29:10.766952 14571 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:29:10.769966 14571 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 14:29:10.770062 14571 log.cc:1196] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 14:29:10.770313 14571 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35: Deleting consensus metadata
I20260808 14:29:10.770503 14571 master_runner.cc:315] Copying system tablet from 127.7.125.126:37133
I20260808 14:29:10.771819 14571 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35: tablet copy: Beginning tablet copy session from remote peer at address 127.7.125.126:37133
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.125.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.124.keytab.
Entry for principal oryx/127.7.125.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.124.keytab.
I20260808 14:29:10.778561 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.124:38861
--webserver_interface=127.7.125.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.124.keytab
--principal=oryx/127.7.125.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:37133,127.7.125.125:33639,127.7.125.124:38861
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:29:10.780984 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:10.772603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55970 (local address 127.7.125.126:37133)
0808 14:29:10.772719 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:10.772724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:10.772969 (+ 245us) server_negotiation.cc:407] Connection header received
0808 14:29:10.773133 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:10.773155 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:10.773218 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:10.773296 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:10.773809 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.774399 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.775557 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.775849 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.775938 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:10.777143 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:10.777146 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:10.777148 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:10.777169 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:10.779036 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:10.779554 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:10.779558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:10.779560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:10.779653 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:10.779981 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:10.779985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:10.779986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:10.780207 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:10.780344 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:10.780737 (+ 393us) server_negotiation.cc:299] Negotiation successful
0808 14:29:10.780866 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 14:29:10.781855 14682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:10.772466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.0.0.1:55970)
0808 14:29:10.772869 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:10.772884 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:10.773007 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:10.773339 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:10.773342 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:10.773552 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:10.773666 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:10.773673 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.774503 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.774508 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:10.775426 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:10.775437 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:10.775964 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:10.775968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:10.776069 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:10.776559 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:10.776573 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:10.776995 (+ 422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:10.779137 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:10.779143 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:10.779148 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:10.779452 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:10.779758 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:10.779764 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:10.779769 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:10.779874 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:10.780350 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:10.780356 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:10.780494 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 14:29:10.781151 (+ 657us) client_negotiation.cc:240] Negotiation successful
0808 14:29:10.781322 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":167,"threads_started":1}
I20260808 14:29:10.782439 14103 tablet_copy_service.cc:140] P d1fbb4311136492580021310c6a777e6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d87c0c9ba2a540778055acacfc93fc35 ({username='oryx', principal='oryx/127.7.125.125@KRBTEST.COM'} at 127.0.0.1:55970)
I20260808 14:29:10.782619 14103 tablet_copy_service.cc:161] P d1fbb4311136492580021310c6a777e6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d87c0c9ba2a540778055acacfc93fc35 at {username='oryx', principal='oryx/127.7.125.125@KRBTEST.COM'} at 127.0.0.1:55970: session id = d87c0c9ba2a540778055acacfc93fc35-00000000000000000000000000000000
I20260808 14:29:10.784396 14103 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 14:29:10.785941 14571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 14:29:10.788043 14571 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35: tablet copy: Starting download of 0 data blocks...
I20260808 14:29:10.788250 14571 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35: tablet copy: Starting download of 1 WAL segments...
I20260808 14:29:10.789899 14571 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 14:29:10.791604 14103 tablet_copy_service.cc:342] P d1fbb4311136492580021310c6a777e6: Request end of tablet copy session d87c0c9ba2a540778055acacfc93fc35-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.125.125@KRBTEST.COM'} at 127.0.0.1:55970
I20260808 14:29:10.791736 14103 tablet_copy_service.cc:433] P d1fbb4311136492580021310c6a777e6: ending tablet copy session d87c0c9ba2a540778055acacfc93fc35-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d87c0c9ba2a540778055acacfc93fc35
I20260808 14:29:10.798502 14571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:10.800105 14684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:10.800398 14571 server_base.cc:1034] running on GCE node
W20260808 14:29:10.800568 14687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:10.807157 14685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:10.807554 14571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:10.807925 14571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:10.809068 14571 hybrid_clock.cc:648] HybridClock initialized: now 1786199350809057 us; error 38 us; skew 500 ppm
Aug 08 14:29:10 dist-test-slave-w206 krb5kdc[14034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199350, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:10.810647 14571 init.cc:377] Logged in from keytab as oryx/127.7.125.125@KRBTEST.COM (short username oryx)
I20260808 14:29:10.811206 14571 webserver.cc:460] Webserver started at http://127.7.125.125:41685/ using document root <none> and password file <none>
I20260808 14:29:10.811421 14571 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:10.811472 14571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:10.812580 14571 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 14:29:10.813508 14695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:10.814129 14571 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 14:29:10.814225 14571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/wal
uuid: "d87c0c9ba2a540778055acacfc93fc35"
format_stamp: "Formatted at 2026-08-08 14:29:10 on dist-test-slave-w206"
server_key: "f3f573f4915fbf50bd0f1b3d77cdaf23"
server_key_iv: "f0a025bd63f3e63377c992e611ee1503"
server_key_version: "encryptionkey@0"
I20260808 14:29:10.814317 14571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:10.841120 14571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:10.843204 14571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:10.848429 14571 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.125:33639
I20260808 14:29:10.848758 14751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.125:33639 every 8 connection(s)
I20260808 14:29:10.849105 14571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-1/data/info.pb
I20260808 14:29:10.850600 14752 sys_catalog.cc:263] Verifying existing consensus state
I20260808 14:29:10.851536 14752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35: Bootstrap starting.
I20260808 14:29:10.859838 14752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:29:10.860168 14752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35: Bootstrap complete.
I20260808 14:29:10.860687 14752 raft_consensus.cc:348] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: NON_VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: true } }
I20260808 14:29:10.860831 14752 raft_consensus.cc:729] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d87c0c9ba2a540778055acacfc93fc35, State: Initialized, Role: LEARNER
I20260808 14:29:10.860951 14752 consensus_queue.cc:260] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: NON_VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: true } }
I20260808 14:29:10.861285 14755 sys_catalog.cc:455] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: NON_VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: true } } }
I20260808 14:29:10.861454 14752 sys_catalog.cc:564] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:29:10.861460 14755 sys_catalog.cc:458] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [sys.catalog]: This master's current role is: LEARNER
I20260808 14:29:10.864216 14768 catalog_manager.cc:1259] Loaded cluster ID: e6e375437ebd404ab1a6a3c7859fe228
I20260808 14:29:10.864537 14768 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35: loading cluster ID for follower catalog manager: success
I20260808 14:29:10.866324 14768 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35: acquiring CA information for follower catalog manager: success
I20260808 14:29:10.867532 14768 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 14:29:10.936709 14683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:10.936964 14683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:10.937003 14683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:10.941073 14683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:29:10.941128 14683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:10.941147 14683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:10.941164 14683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:29:10.941179 14683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:29:10.946846 14683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:37133,127.7.125.125:33639,127.7.125.124:38861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.124.keytab
--principal=oryx/127.7.125.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.124:38861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.125.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:10.947839 14683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:10.948812 14683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:10.957234 14778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:10.957252 14774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:10.957252 14773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:10.957648 14683 server_base.cc:1034] running on GCE node
I20260808 14:29:10.958155 14683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:10.959250 14683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:10.960412 14683 hybrid_clock.cc:648] HybridClock initialized: now 1786199350960399 us; error 24 us; skew 500 ppm
Aug 08 14:29:10 dist-test-slave-w206 krb5kdc[14034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199350, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:10.963440 14683 init.cc:377] Logged in from keytab as oryx/127.7.125.124@KRBTEST.COM (short username oryx)
I20260808 14:29:10.964263 14683 webserver.cc:460] Webserver started at http://127.7.125.124:39393/ using document root <none> and password file <none>
I20260808 14:29:10.964818 14683 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:10.964880 14683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:10.965086 14683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:10.967211 14683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/data/instance:
uuid: "a61f7082b8a84de8b08e73fc67bab028"
format_stamp: "Formatted at 2026-08-08 14:29:10 on dist-test-slave-w206"
server_key: "d36025dce9922bb1825820ed4df8db80"
server_key_iv: "f63e84b3d61901c9070dca182fef524b"
server_key_version: "encryptionkey@0"
I20260808 14:29:10.967823 14683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/wal/instance:
uuid: "a61f7082b8a84de8b08e73fc67bab028"
format_stamp: "Formatted at 2026-08-08 14:29:10 on dist-test-slave-w206"
server_key: "d36025dce9922bb1825820ed4df8db80"
server_key_iv: "f63e84b3d61901c9070dca182fef524b"
server_key_version: "encryptionkey@0"
I20260808 14:29:10.971634 14683 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 14:29:10.973966 14785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:10.975180 14683 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:29:10.975307 14683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/wal
uuid: "a61f7082b8a84de8b08e73fc67bab028"
format_stamp: "Formatted at 2026-08-08 14:29:10 on dist-test-slave-w206"
server_key: "d36025dce9922bb1825820ed4df8db80"
server_key_iv: "f63e84b3d61901c9070dca182fef524b"
server_key_version: "encryptionkey@0"
I20260808 14:29:10.975421 14683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:10.996512 14683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:11.000061 14683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:11.000283 14683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:11.007269 14683 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.124:38861
I20260808 14:29:11.007565 14841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.124:38861 every 8 connection(s)
I20260808 14:29:11.008306 14683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/data/info.pb
I20260808 14:29:11.011262 14842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:11.012805 14284 heartbeater.cc:503] Master 127.7.125.126:37133 was elected leader, sending a full tablet report...
I20260808 14:29:11.015508 14842 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:29:11.018159 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 14683
Aug 08 14:29:11 dist-test-slave-w206 krb5kdc[14034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199350, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.124@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM
Aug 08 14:29:11 dist-test-slave-w206 krb5kdc[14034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.125.124@KRBTEST.COM
I20260808 14:29:11.029188 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.017461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55976 (local address 127.7.125.126:37133)
0808 14:29:11.017610 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.017615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.018456 (+ 841us) server_negotiation.cc:407] Connection header received
0808 14:29:11.019627 (+ 1171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.019630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.019677 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.019742 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:11.020806 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.021326 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.022417 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.022632 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.022738 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.025298 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:11.025301 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:11.025304 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:11.025327 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:11.026911 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.027490 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.027494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.027496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.027605 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.027961 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.027968 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.027970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.028207 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:11.028318 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.028863 (+ 545us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.029002 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:29:11.030144 14845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.019072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57634 (local address 127.7.125.124:38861)
0808 14:29:11.019423 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.019434 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.019473 (+ 39us) server_negotiation.cc:407] Connection header received
0808 14:29:11.019612 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.019617 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.019842 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.020020 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:11.020705 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.021700 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.022499 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.022876 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.023024 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.025716 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:11.025724 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:11.025739 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:11.025767 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:11.027523 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.027974 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.027980 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.027981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.028059 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.028330 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.028333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.028334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.028627 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:11.028830 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.029027 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.029197 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":247,"thread_start_us":154,"threads_started":1}
I20260808 14:29:11.030144 14844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.017821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.0.0.1:55976)
0808 14:29:11.018284 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:11.018347 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:11.019412 (+ 1065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:11.019828 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:11.019837 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:11.020298 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:11.020656 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.020677 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.021444 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.021447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.022291 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.022299 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.022776 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.022780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.022944 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.023585 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:11.023601 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:11.025130 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:11.027038 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.027048 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.027058 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.027394 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.027704 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.027706 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.027708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.027845 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.028331 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:11.028337 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:11.028627 (+ 290us) client_negotiation.cc:769] Sending connection context
0808 14:29:11.028801 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 14:29:11.029149 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":35,"spinlock_wait_cycles":16512,"thread_start_us":135,"threads_started":1}
I20260808 14:29:11.032137 14842 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 14:29:11 dist-test-slave-w206 krb5kdc[14034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199350, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.124@KRBTEST.COM for oryx/127.7.125.125@KRBTEST.COM
I20260808 14:29:11.040226 14844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.032668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:33639 (local address 127.0.0.1:49654)
0808 14:29:11.032806 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:11.032823 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:11.032879 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:11.033255 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:11.033258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:11.033423 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:11.033513 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.033518 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.034391 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.034397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.034921 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.034928 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.035437 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.035441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.035545 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.035937 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:11.035948 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:11.037090 (+ 1142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:11.038921 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.038923 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.038925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.039157 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.039393 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.039395 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.039396 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.039470 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.039786 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:11.039791 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:11.039927 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 14:29:11.040013 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 14:29:11.040111 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 14:29:11.040284 14846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.032765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49654 (local address 127.7.125.125:33639)
0808 14:29:11.033060 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.033063 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.033074 (+ 11us) server_negotiation.cc:407] Connection header received
0808 14:29:11.033108 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.033110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.033147 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.033243 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:11.033624 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.034278 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.035095 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.035330 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.035437 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.037226 (+ 1789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:11.037229 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:11.037231 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:11.037254 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:11.038835 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.039239 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.039241 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.039242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.039315 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.039530 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.039531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.039532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.039685 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:11.039754 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.040070 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.040167 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":129,"threads_started":1}
I20260808 14:29:11.041153 14842 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 14:29:11 dist-test-slave-w206 krb5kdc[14034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199350, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.124@KRBTEST.COM for oryx/127.7.125.124@KRBTEST.COM
I20260808 14:29:11.049425 14844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.041614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:38861 (local address 127.0.0.1:57648)
0808 14:29:11.041723 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:11.041737 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:11.041802 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:11.042021 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:11.042023 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:11.042209 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:11.042300 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.042307 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.043428 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.043431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.044291 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.044318 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.044920 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.044923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.045047 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.045530 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:11.045546 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:11.046685 (+ 1139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:11.048255 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.048257 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.048258 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.048440 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.048671 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.048673 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.048674 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.048742 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.049039 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:11.049041 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:11.049108 (+ 67us) client_negotiation.cc:769] Sending connection context
0808 14:29:11.049183 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 14:29:11.049278 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":24}
I20260808 14:29:11.049479 14845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.041678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57648 (local address 127.7.125.124:38861)
0808 14:29:11.041777 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.041780 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.041791 (+ 11us) server_negotiation.cc:407] Connection header received
0808 14:29:11.041860 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.041861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.041893 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.041987 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:11.042452 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.043270 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.044424 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.044793 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.044916 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.046815 (+ 1899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:11.046818 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:11.046820 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:11.046839 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:11.048162 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.048523 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.048525 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.048526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.048597 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.048812 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.048814 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.048815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.048967 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:11.049039 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.049234 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.049329 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 14:29:11.053085 14842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028: Bootstrap starting.
I20260808 14:29:11.055434 14842 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:11.056152 14842 log.cc:824] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:11.058018 14842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028: No bootstrap required, opened a new log
I20260808 14:29:11.060981 14842 raft_consensus.cc:348] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } }
I20260808 14:29:11.061204 14842 raft_consensus.cc:374] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:11.061285 14842 raft_consensus.cc:729] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a61f7082b8a84de8b08e73fc67bab028, State: Initialized, Role: FOLLOWER
I20260808 14:29:11.061789 14842 consensus_queue.cc:260] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } }
I20260808 14:29:11.062301 14848 sys_catalog.cc:455] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } } }
I20260808 14:29:11.062472 14848 sys_catalog.cc:458] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:29:11.063069 14842 sys_catalog.cc:564] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 14:29:11.067796 14859 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:29:11.067876 14859 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:29:11.071373 14683 master_runner.cc:428] Detected that this master a61f7082b8a84de8b08e73fc67bab028 is joining an existing cluster
I20260808 14:29:11.071436 14683 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.125.124:38861
I20260808 14:29:11.083141 14867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.073799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:38861 (local address 127.0.0.1:57662)
0808 14:29:11.074769 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:11.074782 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:11.074903 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:11.075257 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:11.075260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:11.075528 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:11.075697 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.075705 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.076683 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.076686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.077640 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.077651 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.078360 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.078364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.078459 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.079042 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:11.079058 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:11.079689 (+ 631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:11.081468 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.081472 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.081474 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.081748 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.081996 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.082000 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.082002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.082095 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.082421 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:11.082424 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:11.082540 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 14:29:11.082716 (+ 176us) client_negotiation.cc:240] Negotiation successful
0808 14:29:11.082873 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":897,"mutex_wait_us":74,"spinlock_wait_cycles":3968,"thread_start_us":102,"threads_started":1}
I20260808 14:29:11.083159 14845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.074349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57662 (local address 127.7.125.124:38861)
0808 14:29:11.074441 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.074446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.074847 (+ 401us) server_negotiation.cc:407] Connection header received
0808 14:29:11.075035 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.075038 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.075075 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.075120 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:11.075824 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.076559 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.077776 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.078110 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.078182 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.079838 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:11.079842 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:11.079845 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:11.079867 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:11.081342 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.081839 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.081841 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.081842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.081914 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.082183 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.082185 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.082187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.082337 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:11.082419 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.082776 (+ 357us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.083011 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":30}
I20260808 14:29:11.084004 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 14147
I20260808 14:29:11.094939 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:35815
--local_ip_for_outbound_sockets=127.7.125.65
--tserver_master_addrs=127.7.125.126:37133
--webserver_port=38209
--webserver_interface=127.7.125.65
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.65.keytab
--principal=oryx/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.125.126:37133,127.7.125.125:33639
--tserver_master_addrs=127.7.125.126:37133,127.7.125.125:33639,127.7.125.124:38861 with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:29:11.099653 14846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.074269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49668 (local address 127.7.125.125:33639)
0808 14:29:11.074386 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.074402 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.074859 (+ 457us) server_negotiation.cc:407] Connection header received
0808 14:29:11.075066 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.075072 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.075126 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.075178 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:11.075879 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.076623 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.078574 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.078930 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.079059 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.095651 (+ 16592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:11.095655 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:11.095659 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:11.095693 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:11.097537 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.098113 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.098118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.098121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.098216 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.098574 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.098579 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.098582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.098844 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:11.098933 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.099361 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.099490 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 14:29:11.099671 14866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.074486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:33639 (local address 127.0.0.1:49668)
0808 14:29:11.074788 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:11.074801 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:11.074921 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:11.075311 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:11.075314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:11.075584 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:11.075750 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.075758 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.076754 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.076757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.078441 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.078453 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.079205 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.079208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.079294 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.079786 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:11.079801 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:11.095421 (+ 15620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:11.097695 (+ 2274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.097700 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.097704 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.097988 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.098337 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.098341 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.098345 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.098464 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.099036 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:11.099040 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:11.099155 (+ 115us) client_negotiation.cc:769] Sending connection context
0808 14:29:11.099300 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 14:29:11.099478 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":55,"spinlock_wait_cycles":384,"thread_start_us":87,"threads_started":1}
I20260808 14:29:11.100939 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.074205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55980 (local address 127.7.125.126:37133)
0808 14:29:11.074477 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.074482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.074495 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:29:11.074541 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.074545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.074601 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.074684 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:11.075425 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.076250 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.079062 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.079397 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.079812 (+ 415us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.096614 (+ 16802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:11.096618 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:11.096621 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:11.096654 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:11.098436 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.099011 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.099015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.099017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.099108 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.099413 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.099416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.099418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.099628 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:11.099707 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.100629 (+ 922us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.100776 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191}
I20260808 14:29:11.101451 14865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.073779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.0.0.1:55980)
0808 14:29:11.074133 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:11.074148 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:11.074245 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:11.074815 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:11.074821 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:11.075143 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:11.075297 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.075305 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.076381 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.076384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.078573 (+ 2189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.078581 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.080009 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.080013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.080381 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.095909 (+ 15528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:11.095931 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:11.096451 (+ 520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:11.098577 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.098581 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.098583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.098851 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.099223 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.099226 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.099228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.099313 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.100317 (+ 1004us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:11.100320 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:11.100428 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 14:29:11.101091 (+ 663us) client_negotiation.cc:240] Negotiation successful
0808 14:29:11.101258 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":3,"thread_start_us":124,"threads_started":1}
Aug 08 14:29:11 dist-test-slave-w206 krb5kdc[14034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199349, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.126@KRBTEST.COM for oryx/127.7.125.124@KRBTEST.COM
I20260808 14:29:11.116159 14845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.106180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57678 (local address 127.7.125.124:38861)
0808 14:29:11.106295 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.106300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.106424 (+ 124us) server_negotiation.cc:407] Connection header received
0808 14:29:11.106588 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.106591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.106643 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.106736 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:11.107321 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.108102 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.109385 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.109732 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.109842 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.112482 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:11.112486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:11.112490 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:11.112518 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:11.114340 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.114843 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.114847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.114849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.114941 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.115279 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.115283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.115285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.115505 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:11.115603 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.115876 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.116000 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 14:29:11.116163 14668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.106124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:38861 (local address 127.0.0.1:57678)
0808 14:29:11.106340 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:11.106357 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:11.106461 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:11.106768 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:11.106774 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:11.107033 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:11.107177 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.107187 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.108224 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.108230 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.109235 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.109252 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.109864 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.109868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.110001 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.110511 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:11.110526 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:11.112311 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:11.114455 (+ 2144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.114458 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.114461 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.114745 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.115079 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.115082 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.115084 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.115181 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.115603 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:11.115606 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:11.115710 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 14:29:11.115819 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 14:29:11.115979 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260808 14:29:11.117688 14082 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.125.124:38861: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028"
member_type: NON_VOTER
last_known_addr {
host: "127.7.125.124"
port: 38861
}
attrs {
promote: true
}
}
dest_uuid: "d1fbb4311136492580021310c6a777e6"
cas_config_opid_index: 5
I20260808 14:29:11.118362 14082 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: NON_VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: true } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: NON_VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: true } }
I20260808 14:29:11.119506 14671 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a61f7082b8a84de8b08e73fc67bab028 (127.7.125.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: NON_VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: true } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: NON_VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: true } } }
I20260808 14:29:11.120962 14671 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.125.124:38861
I20260808 14:29:11.121131 14671 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d1fbb4311136492580021310c6a777e6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: NON_VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: true } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: NON_VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: true } } }
I20260808 14:29:11.121219 14671 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:11.122030 14813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:11.122437 14683 master.cc:537] Master@127.7.125.124:38861 shutting down...
I20260808 14:29:11.123584 14813 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 1 FOLLOWER]: Refusing update from remote peer d1fbb4311136492580021310c6a777e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 14:29:11.124603 14670 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: NON_VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:29:11.125833 14668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.119655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:33639 (local address 127.0.0.1:49678)
0808 14:29:11.120199 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:11.120214 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:11.120296 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:11.120708 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:11.120711 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:11.120729 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:11.120823 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.120829 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.122790 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.122793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.124207 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.124219 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.125421 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.125424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.125513 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.125560 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 14:29:11.125632 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 14:29:11.125708 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477}
I20260808 14:29:11.126263 14846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.119888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49678 (local address 127.7.125.125:33639)
0808 14:29:11.120111 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.120116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.120369 (+ 253us) server_negotiation.cc:407] Connection header received
0808 14:29:11.120422 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.120424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.120490 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.120579 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:11.120961 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.121773 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.124360 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.125105 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.125209 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.125357 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.126076 (+ 719us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.126155 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
W20260808 14:29:11.126314 14066 consensus_peers.cc:487] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 -> Peer a61f7082b8a84de8b08e73fc67bab028 (127.7.125.124:38861): Couldn't send request to peer a61f7082b8a84de8b08e73fc67bab028. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 14:29:11.127122 14723 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 1 LEARNER]: Refusing update from remote peer d1fbb4311136492580021310c6a777e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 14:29:11.127532 14670 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: NON_VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 14:29:11.129467 14723 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a61f7082b8a84de8b08e73fc67bab028 (127.7.125.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: NON_VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: true } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: NON_VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: true } } }
I20260808 14:29:11.131752 14755 sys_catalog.cc:455] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1fbb4311136492580021310c6a777e6. Latest consensus state: current_term: 1 leader_uuid: "d1fbb4311136492580021310c6a777e6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: NON_VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: true } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: NON_VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: true } } }
I20260808 14:29:11.131866 14755 sys_catalog.cc:458] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [sys.catalog]: This master's current role is: LEARNER
I20260808 14:29:11.131968 14755 sys_catalog.cc:455] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1fbb4311136492580021310c6a777e6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: NON_VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: true } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: NON_VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: true } } }
I20260808 14:29:11.132031 14755 sys_catalog.cc:458] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [sys.catalog]: This master's current role is: LEARNER
I20260808 14:29:11.132915 14670 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6: attempting to promote NON_VOTER d87c0c9ba2a540778055acacfc93fc35 to VOTER
I20260808 14:29:11.133339 14670 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: NON_VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: true } }
I20260808 14:29:11.134189 14683 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:29:11.134485 14683 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:11.134580 14683 tablet_replica.cc:330] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028: stopping tablet replica
I20260808 14:29:11.134832 14722 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 1 LEARNER]: Refusing update from remote peer d1fbb4311136492580021310c6a777e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 14:29:11.135236 14671 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260808 14:29:11.136881 14066 consensus_peers.cc:487] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 -> Peer a61f7082b8a84de8b08e73fc67bab028 (127.7.125.124:38861): Couldn't send request to peer a61f7082b8a84de8b08e73fc67bab028. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 14:29:11.137161 14671 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d87c0c9ba2a540778055acacfc93fc35 (127.7.125.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: NON_VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: true } } }
I20260808 14:29:11.138690 14671 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d1fbb4311136492580021310c6a777e6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: NON_VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: true } } }
I20260808 14:29:11.138782 14671 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:11.138911 14671 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d1fbb4311136492580021310c6a777e6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: NON_VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: true } } }
I20260808 14:29:11.139041 14671 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:11.139725 14722 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d87c0c9ba2a540778055acacfc93fc35 (127.7.125.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: NON_VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: true } } }
I20260808 14:29:11.141031 14683 master.cc:552] Master@127.7.125.124:38861 shutdown complete.
I20260808 14:29:11.141073 14683 master_runner.cc:305] Clearing existing system tablet
I20260808 14:29:11.141310 14683 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:11.141356 14683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:11.141425 14755 sys_catalog.cc:455] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1fbb4311136492580021310c6a777e6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: NON_VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: true } } }
I20260808 14:29:11.141530 14755 sys_catalog.cc:458] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:29:11.147584 14683 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.002s sys 0.000s
I20260808 14:29:11.148640 14873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:11.148866 14683 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:11.149021 14683 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 14:29:11.149065 14683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/wal
uuid: "a61f7082b8a84de8b08e73fc67bab028"
format_stamp: "Formatted at 2026-08-08 14:29:10 on dist-test-slave-w206"
server_key: "d36025dce9922bb1825820ed4df8db80"
server_key_iv: "f63e84b3d61901c9070dca182fef524b"
server_key_version: "encryptionkey@0"
I20260808 14:29:11.151402 14683 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:29:11.154028 14683 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 14:29:11.154145 14683 log.cc:1196] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 14:29:11.154422 14683 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028: Deleting consensus metadata
I20260808 14:29:11.154592 14683 master_runner.cc:315] Copying system tablet from 127.7.125.126:37133
I20260808 14:29:11.155923 14683 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028: tablet copy: Beginning tablet copy session from remote peer at address 127.7.125.126:37133
I20260808 14:29:11.165995 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.156696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55990 (local address 127.7.125.126:37133)
0808 14:29:11.156878 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.156886 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.156956 (+ 70us) server_negotiation.cc:407] Connection header received
0808 14:29:11.157111 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.157117 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.157188 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.157296 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:11.157863 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.158671 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.160167 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.160504 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.160620 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.162078 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:11.162082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:11.162085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:11.162114 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:11.163984 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.164525 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.164529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.164531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.164630 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.165040 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.165043 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.165045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.165279 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:11.165380 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.165691 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.165832 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 14:29:11.166011 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.156541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.0.0.1:55990)
0808 14:29:11.156871 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:11.156886 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:11.157003 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:11.157332 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:11.157335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:11.157567 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:11.157713 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.157730 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.159063 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.159067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.160022 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.160036 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.160640 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.160643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.160747 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.161349 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:11.161365 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:11.161870 (+ 505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:11.164121 (+ 2251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.164124 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.164127 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.164403 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.164806 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.164827 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.164830 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.164933 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.165379 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:11.165382 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:11.165511 (+ 129us) client_negotiation.cc:769] Sending connection context
0808 14:29:11.165621 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 14:29:11.165790 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":86,"threads_started":1}
I20260808 14:29:11.167042 14103 tablet_copy_service.cc:140] P d1fbb4311136492580021310c6a777e6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a61f7082b8a84de8b08e73fc67bab028 ({username='oryx', principal='oryx/127.7.125.124@KRBTEST.COM'} at 127.0.0.1:55990)
I20260808 14:29:11.167174 14103 tablet_copy_service.cc:161] P d1fbb4311136492580021310c6a777e6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a61f7082b8a84de8b08e73fc67bab028 at {username='oryx', principal='oryx/127.7.125.124@KRBTEST.COM'} at 127.0.0.1:55990: session id = a61f7082b8a84de8b08e73fc67bab028-00000000000000000000000000000000
I20260808 14:29:11.168292 14103 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 14:29:11.169751 14683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 14:29:11.172564 14683 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028: tablet copy: Starting download of 0 data blocks...
I20260808 14:29:11.172894 14683 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028: tablet copy: Starting download of 1 WAL segments...
I20260808 14:29:11.175676 14683 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 14:29:11.177748 14103 tablet_copy_service.cc:342] P d1fbb4311136492580021310c6a777e6: Request end of tablet copy session a61f7082b8a84de8b08e73fc67bab028-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.125.124@KRBTEST.COM'} at 127.0.0.1:55990
I20260808 14:29:11.177852 14103 tablet_copy_service.cc:433] P d1fbb4311136492580021310c6a777e6: ending tablet copy session a61f7082b8a84de8b08e73fc67bab028-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a61f7082b8a84de8b08e73fc67bab028
I20260808 14:29:11.185251 14683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:11.187107 14880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:11.187438 14683 server_base.cc:1034] running on GCE node
W20260808 14:29:11.187134 14883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:11.187898 14881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:11.188226 14683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:11.188541 14683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:11.189694 14683 hybrid_clock.cc:648] HybridClock initialized: now 1786199351189681 us; error 29 us; skew 500 ppm
Aug 08 14:29:11 dist-test-slave-w206 krb5kdc[14034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199351, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:11.191567 14683 init.cc:377] Logged in from keytab as oryx/127.7.125.124@KRBTEST.COM (short username oryx)
I20260808 14:29:11.192090 14683 webserver.cc:460] Webserver started at http://127.7.125.124:36767/ using document root <none> and password file <none>
I20260808 14:29:11.192291 14683 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:11.192342 14683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:11.193819 14683 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 14:29:11.194880 14892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:11.195219 14683 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 14:29:11.195297 14683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/wal
uuid: "a61f7082b8a84de8b08e73fc67bab028"
format_stamp: "Formatted at 2026-08-08 14:29:10 on dist-test-slave-w206"
server_key: "d36025dce9922bb1825820ed4df8db80"
server_key_iv: "f63e84b3d61901c9070dca182fef524b"
server_key_version: "encryptionkey@0"
I20260808 14:29:11.195381 14683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:11.206118 14683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:11.208407 14683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:11.214241 14683 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.124:38861
I20260808 14:29:11.214308 14948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.124:38861 every 8 connection(s)
I20260808 14:29:11.214928 14683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/master-2/data/info.pb
I20260808 14:29:11.216949 14949 sys_catalog.cc:263] Verifying existing consensus state
I20260808 14:29:11.218036 14949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028: Bootstrap starting.
I20260808 14:29:11.230647 14949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:29:11.231063 14949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028: Bootstrap complete.
I20260808 14:29:11.231659 14949 raft_consensus.cc:348] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: NON_VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: true } }
I20260808 14:29:11.231827 14949 raft_consensus.cc:729] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a61f7082b8a84de8b08e73fc67bab028, State: Initialized, Role: LEARNER
I20260808 14:29:11.231948 14949 consensus_queue.cc:260] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: NON_VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: true } }
I20260808 14:29:11.232316 14952 sys_catalog.cc:455] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: NON_VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: true } } }
I20260808 14:29:11.232438 14949 sys_catalog.cc:564] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:29:11.232532 14952 sys_catalog.cc:458] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [sys.catalog]: This master's current role is: LEARNER
I20260808 14:29:11.235812 14965 catalog_manager.cc:1259] Loaded cluster ID: e6e375437ebd404ab1a6a3c7859fe228
I20260808 14:29:11.235937 14965 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028: loading cluster ID for follower catalog manager: success
I20260808 14:29:11.238082 14965 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028: acquiring CA information for follower catalog manager: success
I20260808 14:29:11.239660 14965 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 14:29:11.241312 14423 heartbeater.cc:503] Master 127.7.125.126:37133 was elected leader, sending a full tablet report...
W20260808 14:29:11.244577 14868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:11.244853 14868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:11.244935 14868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:11.249017 14868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:11.249096 14868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:11.249191 14868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:29:11.254547 14868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.65.keytab
--principal=oryx/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:35815
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=38209
--tserver_master_addrs=127.7.125.126:37133,127.7.125.125:33639,127.7.125.124:38861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:11.255673 14868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:11.256695 14868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:11.266407 14971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:11.266367 14975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:11.266433 14972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:11.266583 14868 server_base.cc:1034] running on GCE node
I20260808 14:29:11.267078 14868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:11.267679 14868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:11.268862 14868 hybrid_clock.cc:648] HybridClock initialized: now 1786199351268832 us; error 44 us; skew 500 ppm
Aug 08 14:29:11 dist-test-slave-w206 krb5kdc[14034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199351, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:11.271603 14868 init.cc:377] Logged in from keytab as oryx/127.7.125.65@KRBTEST.COM (short username oryx)
I20260808 14:29:11.272471 14868 webserver.cc:460] Webserver started at http://127.7.125.65:38209/ using document root <none> and password file <none>
I20260808 14:29:11.273029 14868 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:11.273140 14868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:11.276997 14868 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:29:11.279063 14982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:11.280287 14868 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:29:11.280417 14868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "77f51c177c5f49679dfb01ce597d07c8"
format_stamp: "Formatted at 2026-08-08 14:29:09 on dist-test-slave-w206"
server_key: "91027b6978d441bc479f09824bd40e5b"
server_key_iv: "8567d249351156327ebb25a1249c3074"
server_key_version: "encryptionkey@0"
I20260808 14:29:11.280951 14868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:11.299417 14868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:11.302837 14868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:11.303093 14868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:11.303736 14868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:11.304652 14868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:11.304726 14868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:11.304800 14868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:11.304842 14868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:11.314365 15101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:35815 every 8 connection(s)
I20260808 14:29:11.314416 14868 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:35815
I20260808 14:29:11.315650 14868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:29:11.322255 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 14868
I20260808 14:29:11.322479 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 14289
Aug 08 14:29:11 dist-test-slave-w206 krb5kdc[14034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199351, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.124@KRBTEST.COM
Aug 08 14:29:11 dist-test-slave-w206 krb5kdc[14034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199351, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM
I20260808 14:29:11.332744 15106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.317679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:33627 (local address 127.7.125.124:38861)
0808 14:29:11.317956 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.317961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.318789 (+ 828us) server_negotiation.cc:407] Connection header received
0808 14:29:11.320026 (+ 1237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.320030 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.320089 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.320178 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:11.323036 (+ 2858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.323821 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.324967 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.325214 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.325323 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.328530 (+ 3207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:11.328533 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:11.328536 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:11.328566 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:11.330055 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.330700 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.330704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.330706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.330803 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.331281 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.331287 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.331290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.331572 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:11.331650 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.332381 (+ 731us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.332551 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":78,"threads_started":1}
I20260808 14:29:11.334553 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.319359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:55701 (local address 127.7.125.126:37133)
0808 14:29:11.319498 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.319501 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.319583 (+ 82us) server_negotiation.cc:407] Connection header received
0808 14:29:11.320050 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.320054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.320107 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.320162 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:11.321505 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.322042 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.324097 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.324446 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.324554 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.330134 (+ 5580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:11.330138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:11.330142 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:11.330178 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:11.332026 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.332642 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.332646 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.332648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.332732 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.333088 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.333091 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.333093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.333334 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:11.333423 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.334254 (+ 831us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.334381 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 14:29:11.335111 15108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.318142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:38861 (local address 127.7.125.65:33627)
0808 14:29:11.318635 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:11.318690 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:11.319817 (+ 1127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:11.320348 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:11.320355 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:11.320856 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:11.321346 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.321360 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.323937 (+ 2577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.323942 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.324849 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.324857 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.325325 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.325329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.325436 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.325994 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:11.326016 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:11.328355 (+ 2339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:11.330188 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.330194 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.330212 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.330546 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.330933 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.330938 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.330940 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.331156 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.331744 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:11.331754 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:11.332083 (+ 329us) client_negotiation.cc:769] Sending connection context
0808 14:29:11.333258 (+ 1175us) client_negotiation.cc:240] Negotiation successful
0808 14:29:11.333646 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":84,"spinlock_wait_cycles":41600,"thread_start_us":140,"threads_started":1}
Aug 08 14:29:11 dist-test-slave-w206 krb5kdc[14034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199351, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.125@KRBTEST.COM
I20260808 14:29:11.335168 15109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.319205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.65:55701)
0808 14:29:11.319490 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:11.319503 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:11.319822 (+ 319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:11.320349 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:11.320357 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:11.320885 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:11.321349 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.321363 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.322181 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.322185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.323958 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.323974 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.324624 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.324634 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.325011 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.325980 (+ 969us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:11.326012 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:11.329947 (+ 3935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:11.332151 (+ 2204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.332153 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.332156 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.332460 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.332851 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.332854 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.332855 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.332962 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.333584 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:11.333590 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:11.333787 (+ 197us) client_negotiation.cc:769] Sending connection context
0808 14:29:11.334787 (+ 1000us) client_negotiation.cc:240] Negotiation successful
0808 14:29:11.334956 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":188,"spinlock_wait_cycles":35968,"thread_start_us":112,"threads_started":1}
I20260808 14:29:11.337558 15102 heartbeater.cc:347] Connected to a master server at 127.7.125.124:38861
I20260808 14:29:11.337576 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:44269
--local_ip_for_outbound_sockets=127.7.125.66
--tserver_master_addrs=127.7.125.126:37133
--webserver_port=43607
--webserver_interface=127.7.125.66
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.66.keytab
--principal=oryx/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.125.126:37133,127.7.125.125:33639
--tserver_master_addrs=127.7.125.126:37133,127.7.125.125:33639,127.7.125.124:38861 with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:29:11.337850 15102 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:11.337967 15103 heartbeater.cc:347] Connected to a master server at 127.7.125.126:37133
I20260808 14:29:11.338047 15103 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:11.338526 15102 heartbeater.cc:511] Master 127.7.125.124:38861 requested a full tablet report, sending...
I20260808 14:29:11.339243 15103 heartbeater.cc:511] Master 127.7.125.126:37133 requested a full tablet report, sending...
I20260808 14:29:11.340109 14846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.317592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:51165 (local address 127.7.125.125:33639)
0808 14:29:11.317755 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.317761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.318791 (+ 1030us) server_negotiation.cc:407] Connection header received
0808 14:29:11.320050 (+ 1259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.320054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.320110 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.320162 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:11.322170 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.322863 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.332795 (+ 9932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.333272 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.333381 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.336041 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:11.336047 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:11.336053 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:11.336088 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:11.337846 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.338705 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.338711 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.338716 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.338820 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.339161 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.339167 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.339172 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.339449 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:11.339581 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.339824 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.339974 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 14:29:11.340116 15107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.318142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:33639 (local address 127.7.125.65:51165)
0808 14:29:11.318635 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:11.318690 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:11.319816 (+ 1126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:11.320349 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:11.320357 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:11.320885 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:11.321349 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.321363 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.331563 (+ 10200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.331569 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.332637 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.332651 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.333574 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.333581 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.333739 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.334217 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:11.334236 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:11.335875 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:11.338341 (+ 2466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.338344 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.338345 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.338581 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.338923 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.338926 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.338929 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.339053 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.339548 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:11.339551 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:11.339642 (+ 91us) client_negotiation.cc:769] Sending connection context
0808 14:29:11.339744 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 14:29:11.339907 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":98,"spinlock_wait_cycles":25088,"thread_start_us":127,"threads_started":1}
I20260808 14:29:11.340395 14082 ts_manager.cc:194] Re-registered known tserver with Master: 77f51c177c5f49679dfb01ce597d07c8 (127.7.125.65:35815)
I20260808 14:29:11.340705 14909 ts_manager.cc:194] Registered new tserver with Master: 77f51c177c5f49679dfb01ce597d07c8 (127.7.125.65:35815)
I20260808 14:29:11.341066 14082 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:55701
I20260808 14:29:11.341647 15104 heartbeater.cc:347] Connected to a master server at 127.7.125.125:33639
I20260808 14:29:11.341758 15104 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:11.341971 15104 heartbeater.cc:511] Master 127.7.125.125:33639 requested a full tablet report, sending...
I20260808 14:29:11.343830 14713 ts_manager.cc:194] Registered new tserver with Master: 77f51c177c5f49679dfb01ce597d07c8 (127.7.125.65:35815)
W20260808 14:29:11.455183 15110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:11.455447 15110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:11.455520 15110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 14:29:11.456000 14566 heartbeater.cc:503] Master 127.7.125.126:37133 was elected leader, sending a full tablet report...
W20260808 14:29:11.459831 15110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:11.459905 15110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:11.460009 15110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:29:11.465566 15110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.66.keytab
--principal=oryx/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:44269
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=43607
--tserver_master_addrs=127.7.125.126:37133,127.7.125.125:33639,127.7.125.124:38861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:11.466624 15110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:11.467682 15110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:11.477845 15115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:11.477826 15118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:11.477828 15116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:11.477959 15110 server_base.cc:1034] running on GCE node
I20260808 14:29:11.478600 15110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:11.479315 15110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:11.480490 15110 hybrid_clock.cc:648] HybridClock initialized: now 1786199351480458 us; error 45 us; skew 500 ppm
Aug 08 14:29:11 dist-test-slave-w206 krb5kdc[14034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199351, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:11.483322 15110 init.cc:377] Logged in from keytab as oryx/127.7.125.66@KRBTEST.COM (short username oryx)
I20260808 14:29:11.484253 15110 webserver.cc:460] Webserver started at http://127.7.125.66:43607/ using document root <none> and password file <none>
I20260808 14:29:11.484850 15110 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:11.484957 15110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:11.488972 15110 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:29:11.491014 15126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:11.492333 15110 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 14:29:11.492480 15110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "6907de3a9a4144acbce31a6b7f0d6200"
format_stamp: "Formatted at 2026-08-08 14:29:10 on dist-test-slave-w206"
server_key: "d6d37bb3a022013dd9a9d92fe14be22a"
server_key_iv: "0e74222b893b6768982033fe4df76e7d"
server_key_version: "encryptionkey@0"
I20260808 14:29:11.493011 15110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:11.515501 15110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:11.518687 15110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:11.518913 15110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:11.519517 15110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:11.520391 15110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:11.520457 15110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:11.520527 15110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:11.520565 15110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:11.529253 15110 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:44269
I20260808 14:29:11.529312 15242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:44269 every 8 connection(s)
I20260808 14:29:11.530447 15110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:29:11.536782 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 15110
I20260808 14:29:11.536960 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 14428
Aug 08 14:29:11 dist-test-slave-w206 krb5kdc[14034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199351, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.125@KRBTEST.COM
Aug 08 14:29:11 dist-test-slave-w206 krb5kdc[14034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199351, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM
I20260808 14:29:11.547531 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:37109
--local_ip_for_outbound_sockets=127.7.125.67
--tserver_master_addrs=127.7.125.126:37133
--webserver_port=45423
--webserver_interface=127.7.125.67
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.67.keytab
--principal=oryx/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.125.126:37133,127.7.125.125:33639
--tserver_master_addrs=127.7.125.126:37133,127.7.125.125:33639,127.7.125.124:38861 with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 14:29:11 dist-test-slave-w206 krb5kdc[14034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199351, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.124@KRBTEST.COM
I20260808 14:29:11.550462 14846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.532223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:60171 (local address 127.7.125.125:33639)
0808 14:29:11.532351 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.532360 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.533580 (+ 1220us) server_negotiation.cc:407] Connection header received
0808 14:29:11.534943 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.534946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.535040 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.535139 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:11.540660 (+ 5521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.541183 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.542308 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.542551 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.542662 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.545825 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:11.545830 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:11.545836 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:11.545867 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:11.547956 (+ 2089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.548550 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.548556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.548560 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.548641 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.549036 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.549042 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.549046 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.549297 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:11.549371 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.550093 (+ 722us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.550256 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 14:29:11.551626 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.533949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:60541 (local address 127.7.125.126:37133)
0808 14:29:11.534073 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.534078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.534243 (+ 165us) server_negotiation.cc:407] Connection header received
0808 14:29:11.534953 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.534957 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.535033 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.535089 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:11.536584 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.537274 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.542602 (+ 5328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.543066 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.543164 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.545888 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:11.545891 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:11.545893 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:11.545930 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:11.548028 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.549661 (+ 1633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.549665 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.549666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.549760 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.550671 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.550692 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.550694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.550897 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:11.551019 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.551297 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.551444 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 14:29:11.552114 15248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.533835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.66:60541)
0808 14:29:11.534100 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:11.534168 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:11.534744 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:11.535321 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:11.535323 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:11.535803 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:11.536428 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.536439 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.541073 (+ 4634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.541079 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.542450 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.542465 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.543347 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.543353 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.543482 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.543949 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:11.543969 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:11.545702 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:11.549297 (+ 3595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.549301 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.549303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.549551 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.549872 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.549875 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.549878 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.549958 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.551023 (+ 1065us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:11.551027 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:11.551115 (+ 88us) client_negotiation.cc:769] Sending connection context
0808 14:29:11.551220 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 14:29:11.551350 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":98,"spinlock_wait_cycles":2048,"thread_start_us":99,"threads_started":1}
I20260808 14:29:11.552742 15247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.532933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:33639 (local address 127.7.125.66:60171)
0808 14:29:11.533431 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:11.533475 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:11.534736 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:11.535313 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:11.535320 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:11.535795 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:11.536415 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.536428 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.541296 (+ 4868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.541303 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.542177 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.542186 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.542706 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.542710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.542896 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.543565 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:11.543582 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:11.545658 (+ 2076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:11.548096 (+ 2438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.548104 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.548122 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.548446 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.548738 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.548743 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.548747 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.548938 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.549456 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:11.549464 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:11.549785 (+ 321us) client_negotiation.cc:769] Sending connection context
0808 14:29:11.550636 (+ 851us) client_negotiation.cc:240] Negotiation successful
0808 14:29:11.551062 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":73,"spinlock_wait_cycles":41984,"thread_start_us":143,"threads_started":1}
I20260808 14:29:11.553757 15244 heartbeater.cc:347] Connected to a master server at 127.7.125.126:37133
I20260808 14:29:11.553972 15245 heartbeater.cc:347] Connected to a master server at 127.7.125.125:33639
I20260808 14:29:11.554014 15244 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:11.554034 15245 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:11.554608 15245 heartbeater.cc:511] Master 127.7.125.125:33639 requested a full tablet report, sending...
I20260808 14:29:11.555090 15244 heartbeater.cc:511] Master 127.7.125.126:37133 requested a full tablet report, sending...
I20260808 14:29:11.555688 15106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.534385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:34737 (local address 127.7.125.124:38861)
0808 14:29:11.534524 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.534529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.534642 (+ 113us) server_negotiation.cc:407] Connection header received
0808 14:29:11.534954 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.534957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.535033 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.535089 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:11.536596 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.537441 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.548104 (+ 10663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.548523 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.548624 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.551445 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:11.551448 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:11.551451 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:11.551480 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:11.553289 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.553798 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.553804 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.553809 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.553916 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.554494 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.554501 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.554505 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.554776 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:11.554860 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.555292 (+ 432us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.555460 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:29:11.555948 14082 ts_manager.cc:194] Re-registered known tserver with Master: 6907de3a9a4144acbce31a6b7f0d6200 (127.7.125.66:44269)
I20260808 14:29:11.556466 15249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.534238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:38861 (local address 127.7.125.66:34737)
0808 14:29:11.534565 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:11.534576 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:11.534744 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:11.535315 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:11.535321 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:11.535802 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:11.536430 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.536439 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.547061 (+ 10622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.547068 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.547939 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.547953 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.548818 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.548825 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.548980 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.549512 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:11.549532 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:11.551272 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:11.553392 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.553398 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.553402 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.553669 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.554253 (+ 584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.554259 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.554264 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.554372 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.554943 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:11.554947 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:11.555078 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 14:29:11.556013 (+ 935us) client_negotiation.cc:240] Negotiation successful
0808 14:29:11.556181 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":136,"spinlock_wait_cycles":13824,"thread_start_us":119,"threads_started":1}
I20260808 14:29:11.556507 14082 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:60541
I20260808 14:29:11.557061 14713 ts_manager.cc:194] Registered new tserver with Master: 6907de3a9a4144acbce31a6b7f0d6200 (127.7.125.66:44269)
I20260808 14:29:11.557194 15243 heartbeater.cc:347] Connected to a master server at 127.7.125.124:38861
I20260808 14:29:11.557276 15243 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:11.557473 15243 heartbeater.cc:511] Master 127.7.125.124:38861 requested a full tablet report, sending...
I20260808 14:29:11.558202 14909 ts_manager.cc:194] Registered new tserver with Master: 6907de3a9a4144acbce31a6b7f0d6200 (127.7.125.66:44269)
I20260808 14:29:11.636906 14668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.632302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:38861 (local address 127.0.0.1:57682)
0808 14:29:11.632463 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:11.632481 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:11.632605 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:11.632939 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:11.632945 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:11.632972 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:11.633086 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.633094 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.633935 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.633939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.635293 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.635311 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.636469 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.636472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.636565 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.636588 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 14:29:11.636674 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 14:29:11.636775 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 14:29:11.636906 15106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.632402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57682 (local address 127.7.125.124:38861)
0808 14:29:11.632525 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.632529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.632546 (+ 17us) server_negotiation.cc:407] Connection header received
0808 14:29:11.632753 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.632756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.632810 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.632890 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:11.633207 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.633814 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.635485 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.636315 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.636440 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.636599 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.636688 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.636760 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 14:29:11.637826 14919 raft_consensus.cc:1203] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 14:29:11.638278 14952 sys_catalog.cc:455] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1fbb4311136492580021310c6a777e6. Latest consensus state: current_term: 1 leader_uuid: "d1fbb4311136492580021310c6a777e6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: NON_VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: true } } }
I20260808 14:29:11.638446 14952 sys_catalog.cc:458] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [sys.catalog]: This master's current role is: LEARNER
W20260808 14:29:11.671077 15250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:11.671332 15250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:11.671383 15250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:11.675330 15250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:11.675381 15250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:11.675438 15250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:29:11.680670 15250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.67.keytab
--principal=oryx/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:37109
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=45423
--tserver_master_addrs=127.7.125.126:37133,127.7.125.125:33639,127.7.125.124:38861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:11.681645 15250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:11.682595 15250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:11.692466 15256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:11.692459 15255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:11.692589 15250 server_base.cc:1034] running on GCE node
W20260808 14:29:11.692459 15258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:11.693117 15250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:11.693688 15250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:11.694849 15250 hybrid_clock.cc:648] HybridClock initialized: now 1786199351694836 us; error 26 us; skew 500 ppm
Aug 08 14:29:11 dist-test-slave-w206 krb5kdc[14034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199351, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:11.697706 15250 init.cc:377] Logged in from keytab as oryx/127.7.125.67@KRBTEST.COM (short username oryx)
I20260808 14:29:11.698606 15250 webserver.cc:460] Webserver started at http://127.7.125.67:45423/ using document root <none> and password file <none>
I20260808 14:29:11.699272 15250 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:11.699384 15250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:11.703176 15250 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 14:29:11.705155 15266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:11.706348 15250 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:29:11.706475 15250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "fef7bd05dbce458392c26470cccb0bb7"
format_stamp: "Formatted at 2026-08-08 14:29:10 on dist-test-slave-w206"
server_key: "3ce2488e2652450ee27d9b9f791d8788"
server_key_iv: "6c0d30ec675057063cf15286416bf180"
server_key_version: "encryptionkey@0"
I20260808 14:29:11.707032 15250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:11.746692 15250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:11.749598 15250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:11.749812 15250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:11.750406 15250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:11.751289 15250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:11.751360 15250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:11.751432 15250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:11.751474 15250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:11.760308 15250 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:37109
I20260808 14:29:11.760331 15384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:37109 every 8 connection(s)
I20260808 14:29:11.761288 15250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:29:11.768453 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 15250
Aug 08 14:29:11 dist-test-slave-w206 krb5kdc[14034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199351, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM
Aug 08 14:29:11 dist-test-slave-w206 krb5kdc[14034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199351, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.125@KRBTEST.COM
Aug 08 14:29:11 dist-test-slave-w206 krb5kdc[14034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199351, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.124@KRBTEST.COM
I20260808 14:29:11.782409 15391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.764752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.67:42243)
0808 14:29:11.765025 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:11.765037 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:11.765385 (+ 348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:11.765900 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:11.765902 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:11.766395 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:11.766956 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.767009 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.768546 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.768551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.769500 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.769512 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.770125 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.770127 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.770288 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.771245 (+ 957us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:11.771262 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:11.774432 (+ 3170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:11.776571 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.776576 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.776596 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.777663 (+ 1067us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.779401 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.779405 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.779406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.779605 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.780275 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:11.780281 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:11.780653 (+ 372us) client_negotiation.cc:769] Sending connection context
0808 14:29:11.780875 (+ 222us) client_negotiation.cc:240] Negotiation successful
0808 14:29:11.781239 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":69,"spinlock_wait_cycles":2432,"thread_start_us":96,"threads_started":1}
I20260808 14:29:11.783377 15396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.770945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57694 (local address 127.7.125.124:38861)
0808 14:29:11.771607 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.771612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.773948 (+ 2336us) server_negotiation.cc:407] Connection header received
0808 14:29:11.774136 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.774139 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.774191 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.774254 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:11.775283 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.776100 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.777263 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.777623 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.777703 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.779417 (+ 1714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:11.779421 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:11.779423 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:11.779449 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:11.781137 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.781627 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.781630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.781632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.781720 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.782031 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.782034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.782036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.782247 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:11.782357 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.783090 (+ 733us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.783213 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":76,"threads_started":1}
I20260808 14:29:11.783370 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.764857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:42243 (local address 127.7.125.126:37133)
0808 14:29:11.764988 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.764993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.765115 (+ 122us) server_negotiation.cc:407] Connection header received
0808 14:29:11.765585 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.765588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.765648 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.765727 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:11.767811 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.768403 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.769639 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.769903 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.769963 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.774593 (+ 4630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:11.774597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:11.774600 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:11.774634 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:11.776446 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.779159 (+ 2713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.779162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.779165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.779260 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.779821 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.779827 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.779830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.780096 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:11.780212 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.783090 (+ 2878us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.783213 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 14:29:11.783627 15390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.763652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:33639 (local address 127.7.125.67:39579)
0808 14:29:11.764210 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:11.764271 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:11.765367 (+ 1096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:11.765893 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:11.765903 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:11.766401 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:11.766993 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.767002 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.767845 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.767848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.769535 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.769541 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.771167 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.771171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.771256 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.771729 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:11.771743 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:11.773877 (+ 2134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:11.781586 (+ 7709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.781590 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.781592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.781847 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.782153 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.782156 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.782157 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.782248 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.782702 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:11.782705 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:11.782796 (+ 91us) client_negotiation.cc:769] Sending connection context
0808 14:29:11.782899 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 14:29:11.783057 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":328,"spinlock_wait_cycles":33792,"thread_start_us":155,"threads_started":1}
I20260808 14:29:11.783661 15397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.771202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55994 (local address 127.7.125.126:37133)
0808 14:29:11.771755 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.771762 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.772530 (+ 768us) server_negotiation.cc:407] Connection header received
0808 14:29:11.772574 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.772578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.772626 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.772706 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:11.773818 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.774639 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.776805 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.777201 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.777318 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.779456 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:11.779460 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:11.779463 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:11.779488 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:11.781331 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.781886 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.781891 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.781895 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.781988 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.782303 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.782309 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.782314 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.782522 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:11.782652 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.783324 (+ 672us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.783466 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":95,"threads_started":1}
I20260808 14:29:11.784744 14846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.763186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:39579 (local address 127.7.125.125:33639)
0808 14:29:11.763326 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.763331 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.764378 (+ 1047us) server_negotiation.cc:407] Connection header received
0808 14:29:11.765603 (+ 1225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.765606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.765660 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.765716 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:11.767165 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.767691 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.770537 (+ 2846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.770889 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.770966 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.774063 (+ 3097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:11.774067 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:11.774071 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:11.774099 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:11.781331 (+ 7232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.781957 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.781960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.781962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.782047 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.782370 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.782373 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.782375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.782570 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:11.782652 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.784450 (+ 1798us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.784580 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 14:29:11.785475 15386 heartbeater.cc:347] Connected to a master server at 127.7.125.126:37133
I20260808 14:29:11.785809 15387 heartbeater.cc:347] Connected to a master server at 127.7.125.125:33639
I20260808 14:29:11.785912 15387 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:11.785919 15386 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:11.786466 15387 heartbeater.cc:511] Master 127.7.125.125:33639 requested a full tablet report, sending...
I20260808 14:29:11.786487 15386 heartbeater.cc:511] Master 127.7.125.126:37133 requested a full tablet report, sending...
I20260808 14:29:11.787021 15389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.763652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:38861 (local address 127.7.125.67:39683)
0808 14:29:11.764210 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:11.764271 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:11.765367 (+ 1096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:11.765893 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:11.765903 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:11.766401 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:11.766993 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.767002 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.767845 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.767848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.769515 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.769525 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.770243 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.770246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.770339 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.771208 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:11.771230 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:11.776276 (+ 5046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:11.779477 (+ 3201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.779480 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.779482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.785556 (+ 6074us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.785907 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:11.785911 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:11.785912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:11.786009 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:11.786492 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:11.786496 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:11.786580 (+ 84us) client_negotiation.cc:769] Sending connection context
0808 14:29:11.786686 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 14:29:11.786819 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":269,"spinlock_wait_cycles":45952,"thread_start_us":139,"threads_started":1}
I20260808 14:29:11.787111 15106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.763198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:39683 (local address 127.7.125.124:38861)
0808 14:29:11.763443 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.763448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.764399 (+ 951us) server_negotiation.cc:407] Connection header received
0808 14:29:11.765601 (+ 1202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.765605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.765664 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.765716 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:11.767139 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.767684 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.769661 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.769972 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.770053 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.776445 (+ 6392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:11.776448 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:11.776452 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:11.776481 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:11.779359 (+ 2878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.785679 (+ 6320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.785682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.785684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.785778 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.786126 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.786129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.786131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.786331 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:11.786434 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.786802 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.786920 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260808 14:29:11.787628 15385 heartbeater.cc:347] Connected to a master server at 127.7.125.124:38861
I20260808 14:29:11.787714 15385 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:11.787870 15385 heartbeater.cc:511] Master 127.7.125.124:38861 requested a full tablet report, sending...
I20260808 14:29:11.787961 14713 ts_manager.cc:194] Registered new tserver with Master: fef7bd05dbce458392c26470cccb0bb7 (127.7.125.67:37109)
I20260808 14:29:11.788053 14081 ts_manager.cc:194] Re-registered known tserver with Master: fef7bd05dbce458392c26470cccb0bb7 (127.7.125.67:37109)
I20260808 14:29:11.788571 14909 ts_manager.cc:194] Registered new tserver with Master: fef7bd05dbce458392c26470cccb0bb7 (127.7.125.67:37109)
I20260808 14:29:11.788796 14081 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:42243
I20260808 14:29:11.790895 15400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.771920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49682 (local address 127.7.125.125:33639)
0808 14:29:11.775097 (+ 3177us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.775112 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.775125 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:29:11.775160 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.775162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.775214 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.775297 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:11.776193 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.777054 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.778134 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.778527 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.778638 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.784686 (+ 6048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:11.784690 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:11.784692 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:11.784718 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:11.788834 (+ 4116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.789336 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.789339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.789341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.789441 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:11.789742 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:11.789745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:11.789746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:11.789941 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:11.790061 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.790633 (+ 572us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.790776 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3103,"thread_start_us":77,"threads_started":1}
I20260808 14:29:11.789497 14081 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55994:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:29:11.793162 14081 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:29:11.795996 14871 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d1fbb4311136492580021310c6a777e6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: NON_VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: true } } }
I20260808 14:29:11.796180 14871 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:11.799515 15406 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6: attempting to promote NON_VOTER a61f7082b8a84de8b08e73fc67bab028 to VOTER
I20260808 14:29:11.799893 15406 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: false } }
I20260808 14:29:11.801272 14919 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 1 LEARNER]: Refusing update from remote peer d1fbb4311136492580021310c6a777e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 14:29:11.801664 14722 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 1 FOLLOWER]: Refusing update from remote peer d1fbb4311136492580021310c6a777e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 14:29:11.801707 15405 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 14:29:11.802121 15405 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 14:29:11.803632 15406 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, a61f7082b8a84de8b08e73fc67bab028 (127.7.125.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: false } } }
I20260808 14:29:11.804067 14722 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, a61f7082b8a84de8b08e73fc67bab028 (127.7.125.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: false } } }
I20260808 14:29:11.805280 15406 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d1fbb4311136492580021310c6a777e6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: false } } }
I20260808 14:29:11.805408 15406 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:11.805536 15406 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d1fbb4311136492580021310c6a777e6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: false } } }
I20260808 14:29:11.805622 15406 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:11.805600 14871 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d1fbb4311136492580021310c6a777e6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: false } } }
I20260808 14:29:11.805747 14871 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6 [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:11.806393 15415 sys_catalog.cc:455] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1fbb4311136492580021310c6a777e6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: false } } }
I20260808 14:29:11.806564 15415 sys_catalog.cc:458] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:29:11.806228 14919 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, a61f7082b8a84de8b08e73fc67bab028 (127.7.125.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: false } } }
I20260808 14:29:11.808055 14952 sys_catalog.cc:455] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1fbb4311136492580021310c6a777e6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: false } } }
I20260808 14:29:11.808194 14952 sys_catalog.cc:458] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:29:11.816260 14668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.810005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:35815 (local address 127.0.0.1:37184)
0808 14:29:11.810176 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:11.810192 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:11.810271 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:11.811832 (+ 1561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:11.811835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:11.811854 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:11.811971 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.811982 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.813505 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.813511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.814709 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.814720 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.815862 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.815865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.815933 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.815961 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 14:29:11.816006 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 14:29:11.816107 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":7}
I20260808 14:29:11.816315 15420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.811277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48996 (local address 127.7.125.67:37109)
0808 14:29:11.811592 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.811603 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.812292 (+ 689us) server_negotiation.cc:407] Connection header received
0808 14:29:11.812382 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.812387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.812583 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.812754 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:11.812941 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.813657 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.814750 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.815606 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.815717 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.815874 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.816052 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.816167 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":127,"threads_started":1}
I20260808 14:29:11.816871 15416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.810250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:44269 (local address 127.0.0.1:54002)
0808 14:29:11.810888 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:11.810901 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:11.811024 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:11.812058 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:11.812061 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:11.812083 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:11.812176 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.812182 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.813788 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.813791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.815101 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.815111 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.816191 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.816194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.816271 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.816300 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 14:29:11.816670 (+ 370us) client_negotiation.cc:240] Negotiation successful
0808 14:29:11.816744 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":117,"threads_started":1}
I20260808 14:29:11.816974 15418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.810727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54002 (local address 127.7.125.66:44269)
0808 14:29:11.811307 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.811323 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.811348 (+ 25us) server_negotiation.cc:407] Connection header received
0808 14:29:11.811429 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.811435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.811666 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.811851 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:11.812308 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.813433 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.815238 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.816093 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.816198 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.816341 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.816718 (+ 377us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.816819 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":116,"threads_started":1}
I20260808 14:29:11.816994 15417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.810232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37184 (local address 127.7.125.65:35815)
0808 14:29:11.811310 (+ 1078us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:11.811325 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:11.811348 (+ 23us) server_negotiation.cc:407] Connection header received
0808 14:29:11.811429 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:11.811435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:11.811696 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:11.811851 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:11.812116 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.813349 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.814861 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.815735 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.816554 (+ 819us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.816700 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:11.816750 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 14:29:11.816836 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":925,"thread_start_us":118,"threads_started":1}
I20260808 14:29:11.817507 15419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:11.811131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:37109 (local address 127.0.0.1:48996)
0808 14:29:11.812091 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:11.812105 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:11.812194 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:11.812728 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:11.812731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:11.812749 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:11.812846 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.812854 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.813770 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.813776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.814618 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:11.814628 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:11.815751 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:11.815755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:11.815874 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:11.815916 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 14:29:11.815988 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 14:29:11.816109 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":899,"mutex_wait_us":76,"thread_start_us":93,"threads_started":1}
I20260808 14:29:11.819801 15032 tablet_service.cc:1467] Processing CreateTablet for tablet f3032262e5544017a94726588d4f6afe (DEFAULT_TABLE table=test-table [id=f17bc4ba310c4791a5e731ce2038c9e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:11.820218 15174 tablet_service.cc:1467] Processing CreateTablet for tablet f3032262e5544017a94726588d4f6afe (DEFAULT_TABLE table=test-table [id=f17bc4ba310c4791a5e731ce2038c9e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:11.820218 15315 tablet_service.cc:1467] Processing CreateTablet for tablet f3032262e5544017a94726588d4f6afe (DEFAULT_TABLE table=test-table [id=f17bc4ba310c4791a5e731ce2038c9e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:11.820806 15032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3032262e5544017a94726588d4f6afe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:11.821214 15315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3032262e5544017a94726588d4f6afe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:11.821213 15174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3032262e5544017a94726588d4f6afe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:11.827736 15421 tablet_bootstrap.cc:492] T f3032262e5544017a94726588d4f6afe P 77f51c177c5f49679dfb01ce597d07c8: Bootstrap starting.
I20260808 14:29:11.828790 15422 tablet_bootstrap.cc:492] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7: Bootstrap starting.
I20260808 14:29:11.829224 15423 tablet_bootstrap.cc:492] T f3032262e5544017a94726588d4f6afe P 6907de3a9a4144acbce31a6b7f0d6200: Bootstrap starting.
I20260808 14:29:11.830400 15421 tablet_bootstrap.cc:654] T f3032262e5544017a94726588d4f6afe P 77f51c177c5f49679dfb01ce597d07c8: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:11.831337 15421 log.cc:824] T f3032262e5544017a94726588d4f6afe P 77f51c177c5f49679dfb01ce597d07c8: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:11.831498 15423 tablet_bootstrap.cc:654] T f3032262e5544017a94726588d4f6afe P 6907de3a9a4144acbce31a6b7f0d6200: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:11.832191 15423 log.cc:824] T f3032262e5544017a94726588d4f6afe P 6907de3a9a4144acbce31a6b7f0d6200: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:11.833134 15422 tablet_bootstrap.cc:654] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:11.833758 15421 tablet_bootstrap.cc:492] T f3032262e5544017a94726588d4f6afe P 77f51c177c5f49679dfb01ce597d07c8: No bootstrap required, opened a new log
I20260808 14:29:11.833978 15422 log.cc:824] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:11.834350 15421 ts_tablet_manager.cc:1397] T f3032262e5544017a94726588d4f6afe P 77f51c177c5f49679dfb01ce597d07c8: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 14:29:11.834446 15423 tablet_bootstrap.cc:492] T f3032262e5544017a94726588d4f6afe P 6907de3a9a4144acbce31a6b7f0d6200: No bootstrap required, opened a new log
I20260808 14:29:11.834630 15423 ts_tablet_manager.cc:1397] T f3032262e5544017a94726588d4f6afe P 6907de3a9a4144acbce31a6b7f0d6200: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:29:11.835665 15422 tablet_bootstrap.cc:492] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7: No bootstrap required, opened a new log
I20260808 14:29:11.835855 15422 ts_tablet_manager.cc:1397] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 14:29:11.838678 15421 raft_consensus.cc:348] T f3032262e5544017a94726588d4f6afe P 77f51c177c5f49679dfb01ce597d07c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77f51c177c5f49679dfb01ce597d07c8" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35815 } } peers { permanent_uuid: "6907de3a9a4144acbce31a6b7f0d6200" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44269 } } peers { permanent_uuid: "fef7bd05dbce458392c26470cccb0bb7" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37109 } }
I20260808 14:29:11.839006 15421 raft_consensus.cc:374] T f3032262e5544017a94726588d4f6afe P 77f51c177c5f49679dfb01ce597d07c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:11.839097 15421 raft_consensus.cc:729] T f3032262e5544017a94726588d4f6afe P 77f51c177c5f49679dfb01ce597d07c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77f51c177c5f49679dfb01ce597d07c8, State: Initialized, Role: FOLLOWER
I20260808 14:29:11.839293 15423 raft_consensus.cc:348] T f3032262e5544017a94726588d4f6afe P 6907de3a9a4144acbce31a6b7f0d6200 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77f51c177c5f49679dfb01ce597d07c8" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35815 } } peers { permanent_uuid: "6907de3a9a4144acbce31a6b7f0d6200" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44269 } } peers { permanent_uuid: "fef7bd05dbce458392c26470cccb0bb7" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37109 } }
I20260808 14:29:11.839532 15423 raft_consensus.cc:374] T f3032262e5544017a94726588d4f6afe P 6907de3a9a4144acbce31a6b7f0d6200 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:11.839608 15423 raft_consensus.cc:729] T f3032262e5544017a94726588d4f6afe P 6907de3a9a4144acbce31a6b7f0d6200 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6907de3a9a4144acbce31a6b7f0d6200, State: Initialized, Role: FOLLOWER
I20260808 14:29:11.839598 15421 consensus_queue.cc:260] T f3032262e5544017a94726588d4f6afe P 77f51c177c5f49679dfb01ce597d07c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77f51c177c5f49679dfb01ce597d07c8" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35815 } } peers { permanent_uuid: "6907de3a9a4144acbce31a6b7f0d6200" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44269 } } peers { permanent_uuid: "fef7bd05dbce458392c26470cccb0bb7" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37109 } }
I20260808 14:29:11.839898 15422 raft_consensus.cc:348] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77f51c177c5f49679dfb01ce597d07c8" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35815 } } peers { permanent_uuid: "6907de3a9a4144acbce31a6b7f0d6200" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44269 } } peers { permanent_uuid: "fef7bd05dbce458392c26470cccb0bb7" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37109 } }
I20260808 14:29:11.840051 15423 consensus_queue.cc:260] T f3032262e5544017a94726588d4f6afe P 6907de3a9a4144acbce31a6b7f0d6200 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77f51c177c5f49679dfb01ce597d07c8" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35815 } } peers { permanent_uuid: "6907de3a9a4144acbce31a6b7f0d6200" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44269 } } peers { permanent_uuid: "fef7bd05dbce458392c26470cccb0bb7" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37109 } }
I20260808 14:29:11.840186 15422 raft_consensus.cc:374] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:11.840257 15422 raft_consensus.cc:729] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef7bd05dbce458392c26470cccb0bb7, State: Initialized, Role: FOLLOWER
I20260808 14:29:11.840440 15103 heartbeater.cc:503] Master 127.7.125.126:37133 was elected leader, sending a full tablet report...
I20260808 14:29:11.840693 15422 consensus_queue.cc:260] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77f51c177c5f49679dfb01ce597d07c8" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35815 } } peers { permanent_uuid: "6907de3a9a4144acbce31a6b7f0d6200" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44269 } } peers { permanent_uuid: "fef7bd05dbce458392c26470cccb0bb7" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37109 } }
I20260808 14:29:11.841228 15423 ts_tablet_manager.cc:1428] T f3032262e5544017a94726588d4f6afe P 6907de3a9a4144acbce31a6b7f0d6200: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 14:29:11.841581 15422 ts_tablet_manager.cc:1428] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 14:29:11.842334 15386 heartbeater.cc:503] Master 127.7.125.126:37133 was elected leader, sending a full tablet report...
I20260808 14:29:11.844960 15421 ts_tablet_manager.cc:1428] T f3032262e5544017a94726588d4f6afe P 77f51c177c5f49679dfb01ce597d07c8: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260808 14:29:11.845546 15244 heartbeater.cc:503] Master 127.7.125.126:37133 was elected leader, sending a full tablet report...
W20260808 14:29:12.012977 15388 tablet.cc:2369] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:29:12.031853 15246 tablet.cc:2369] T f3032262e5544017a94726588d4f6afe P 6907de3a9a4144acbce31a6b7f0d6200: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:29:12.067934 15105 tablet.cc:2369] T f3032262e5544017a94726588d4f6afe P 77f51c177c5f49679dfb01ce597d07c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:29:12.094129 15430 raft_consensus.cc:482] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:29:12.094360 15430 raft_consensus.cc:504] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77f51c177c5f49679dfb01ce597d07c8" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35815 } } peers { permanent_uuid: "6907de3a9a4144acbce31a6b7f0d6200" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44269 } } peers { permanent_uuid: "fef7bd05dbce458392c26470cccb0bb7" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37109 } }
I20260808 14:29:12.095774 15430 leader_election.cc:288] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 77f51c177c5f49679dfb01ce597d07c8 (127.7.125.65:35815), 6907de3a9a4144acbce31a6b7f0d6200 (127.7.125.66:44269)
I20260808 14:29:12.099884 15417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:12.096182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:58427 (local address 127.7.125.65:35815)
0808 14:29:12.096374 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:12.096381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:12.096396 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:29:12.096470 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:12.096476 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:12.096526 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:12.096614 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:12.096904 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:12.097491 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:12.098726 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:12.099360 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:12.099455 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:12.099496 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:12.099723 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 14:29:12.099781 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 14:29:12.099941 15389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:12.095952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:35815 (local address 127.7.125.67:58427)
0808 14:29:12.096145 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:12.096161 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:12.096253 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:12.096629 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:12.096631 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:12.096645 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:12.096767 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:12.096776 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:12.097606 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:12.097610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:12.098597 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:12.098606 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:12.099476 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:12.099480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:12.099597 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:12.099623 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 14:29:12.099704 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 14:29:12.099772 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260808 14:29:12.100503 15390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:12.096199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:44269 (local address 127.7.125.67:43707)
0808 14:29:12.096297 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:12.096324 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:12.096403 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:12.097130 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:12.097132 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:12.097147 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:12.097271 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:12.097276 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:12.098198 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:12.098200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:12.099185 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:12.099195 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:12.100128 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:12.100131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:12.100201 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:12.100216 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 14:29:12.100316 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 14:29:12.100386 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 14:29:12.100561 15418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:12.096817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:43707 (local address 127.7.125.66:44269)
0808 14:29:12.096940 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:12.096944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:12.096955 (+ 11us) server_negotiation.cc:407] Connection header received
0808 14:29:12.096999 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:12.097001 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:12.097036 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:12.097121 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:12.097390 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:12.098091 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:12.099301 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:12.099886 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:12.100020 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:12.100079 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:12.100346 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 14:29:12.100439 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 14:29:12.100757 15052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3032262e5544017a94726588d4f6afe" candidate_uuid: "fef7bd05dbce458392c26470cccb0bb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77f51c177c5f49679dfb01ce597d07c8" is_pre_election: true
I20260808 14:29:12.101105 15052 raft_consensus.cc:2454] T f3032262e5544017a94726588d4f6afe P 77f51c177c5f49679dfb01ce597d07c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fef7bd05dbce458392c26470cccb0bb7 in term 0.
I20260808 14:29:12.101105 15196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3032262e5544017a94726588d4f6afe" candidate_uuid: "fef7bd05dbce458392c26470cccb0bb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6907de3a9a4144acbce31a6b7f0d6200" is_pre_election: true
I20260808 14:29:12.101392 15196 raft_consensus.cc:2454] T f3032262e5544017a94726588d4f6afe P 6907de3a9a4144acbce31a6b7f0d6200 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fef7bd05dbce458392c26470cccb0bb7 in term 0.
I20260808 14:29:12.101660 15267 leader_election.cc:302] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77f51c177c5f49679dfb01ce597d07c8, fef7bd05dbce458392c26470cccb0bb7; no voters:
I20260808 14:29:12.101969 15430 raft_consensus.cc:2781] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:29:12.102061 15430 raft_consensus.cc:482] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:29:12.102115 15430 raft_consensus.cc:3037] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:12.103196 15430 raft_consensus.cc:504] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77f51c177c5f49679dfb01ce597d07c8" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35815 } } peers { permanent_uuid: "6907de3a9a4144acbce31a6b7f0d6200" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44269 } } peers { permanent_uuid: "fef7bd05dbce458392c26470cccb0bb7" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37109 } }
I20260808 14:29:12.103547 15430 leader_election.cc:288] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7 [CANDIDATE]: Term 1 election: Requested vote from peers 77f51c177c5f49679dfb01ce597d07c8 (127.7.125.65:35815), 6907de3a9a4144acbce31a6b7f0d6200 (127.7.125.66:44269)
I20260808 14:29:12.103878 15052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3032262e5544017a94726588d4f6afe" candidate_uuid: "fef7bd05dbce458392c26470cccb0bb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77f51c177c5f49679dfb01ce597d07c8"
I20260808 14:29:12.103916 15196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3032262e5544017a94726588d4f6afe" candidate_uuid: "fef7bd05dbce458392c26470cccb0bb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6907de3a9a4144acbce31a6b7f0d6200"
I20260808 14:29:12.104014 15052 raft_consensus.cc:3037] T f3032262e5544017a94726588d4f6afe P 77f51c177c5f49679dfb01ce597d07c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:12.104044 15196 raft_consensus.cc:3037] T f3032262e5544017a94726588d4f6afe P 6907de3a9a4144acbce31a6b7f0d6200 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:12.105047 15196 raft_consensus.cc:2454] T f3032262e5544017a94726588d4f6afe P 6907de3a9a4144acbce31a6b7f0d6200 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fef7bd05dbce458392c26470cccb0bb7 in term 1.
I20260808 14:29:12.105106 15052 raft_consensus.cc:2454] T f3032262e5544017a94726588d4f6afe P 77f51c177c5f49679dfb01ce597d07c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fef7bd05dbce458392c26470cccb0bb7 in term 1.
I20260808 14:29:12.105366 15269 leader_election.cc:302] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6907de3a9a4144acbce31a6b7f0d6200, fef7bd05dbce458392c26470cccb0bb7; no voters:
I20260808 14:29:12.105576 15430 raft_consensus.cc:2781] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:29:12.105829 15430 raft_consensus.cc:686] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7 [term 1 LEADER]: Becoming Leader. State: Replica: fef7bd05dbce458392c26470cccb0bb7, State: Running, Role: LEADER
I20260808 14:29:12.106180 15430 consensus_queue.cc:237] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77f51c177c5f49679dfb01ce597d07c8" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35815 } } peers { permanent_uuid: "6907de3a9a4144acbce31a6b7f0d6200" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44269 } } peers { permanent_uuid: "fef7bd05dbce458392c26470cccb0bb7" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37109 } }
I20260808 14:29:12.110081 14082 catalog_manager.cc:5495] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to fef7bd05dbce458392c26470cccb0bb7 (127.7.125.67). New cstate: current_term: 1 leader_uuid: "fef7bd05dbce458392c26470cccb0bb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77f51c177c5f49679dfb01ce597d07c8" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6907de3a9a4144acbce31a6b7f0d6200" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fef7bd05dbce458392c26470cccb0bb7" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37109 } health_report { overall_health: HEALTHY } } }
I20260808 14:29:12.175606 15420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:12.172055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49008 (local address 127.7.125.67:37109)
0808 14:29:12.172192 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:12.172195 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:12.172210 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:29:12.172287 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:12.172290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:12.172344 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:12.172439 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:29:12.172679 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:12.173180 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:12.174003 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:12.174209 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:12.174290 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:12.174631 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:29:12.175100 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:29:12.175195 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:12.175368 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 14:29:12.175423 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:29:12.181577 15196 raft_consensus.cc:1261] T f3032262e5544017a94726588d4f6afe P 6907de3a9a4144acbce31a6b7f0d6200 [term 1 FOLLOWER]: Refusing update from remote peer fef7bd05dbce458392c26470cccb0bb7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:29:12.181577 15052 raft_consensus.cc:1261] T f3032262e5544017a94726588d4f6afe P 77f51c177c5f49679dfb01ce597d07c8 [term 1 FOLLOWER]: Refusing update from remote peer fef7bd05dbce458392c26470cccb0bb7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:29:12.182323 15430 consensus_queue.cc:1035] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6907de3a9a4144acbce31a6b7f0d6200" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:29:12.182546 15431 consensus_queue.cc:1035] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77f51c177c5f49679dfb01ce597d07c8" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:29:12.193603 15435 mvcc.cc:205] Tried to move back new op lower bound from 7316272546528346112 to 7316272546229452800. Current Snapshot: MvccSnapshot[applied={T|T < 7316272546528346112}]
I20260808 14:29:12.196923 15434 mvcc.cc:205] Tried to move back new op lower bound from 7316272546528346112 to 7316272546229452800. Current Snapshot: MvccSnapshot[applied={T|T < 7316272546528346112 or (T in {7316272546528346112})}]
I20260808 14:29:12.197149 15436 mvcc.cc:205] Tried to move back new op lower bound from 7316272546528346112 to 7316272546229452800. Current Snapshot: MvccSnapshot[applied={T|T < 7316272546528346112}]
I20260808 14:29:12.202083 14082 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:29:12.202261 14082 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:29:12.207320 14082 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d1fbb4311136492580021310c6a777e6: Sending DeleteTablet for 3 replicas of tablet f3032262e5544017a94726588d4f6afe
I20260808 14:29:12.208170 15315 tablet_service.cc:1514] Processing DeleteTablet for tablet f3032262e5544017a94726588d4f6afe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:12 UTC) from {username='oryx', principal='oryx/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:48996
I20260808 14:29:12.208684 15174 tablet_service.cc:1514] Processing DeleteTablet for tablet f3032262e5544017a94726588d4f6afe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:12 UTC) from {username='oryx', principal='oryx/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:54002
I20260808 14:29:12.209071 15032 tablet_service.cc:1514] Processing DeleteTablet for tablet f3032262e5544017a94726588d4f6afe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:12 UTC) from {username='oryx', principal='oryx/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:37184
I20260808 14:29:12.209157 15444 tablet_replica.cc:330] T f3032262e5544017a94726588d4f6afe P 6907de3a9a4144acbce31a6b7f0d6200: stopping tablet replica
I20260808 14:29:12.209506 15444 raft_consensus.cc:2229] T f3032262e5544017a94726588d4f6afe P 6907de3a9a4144acbce31a6b7f0d6200 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:29:12.209744 15445 tablet_replica.cc:330] T f3032262e5544017a94726588d4f6afe P 77f51c177c5f49679dfb01ce597d07c8: stopping tablet replica
I20260808 14:29:12.209908 15444 raft_consensus.cc:2258] T f3032262e5544017a94726588d4f6afe P 6907de3a9a4144acbce31a6b7f0d6200 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:12.210500 15445 raft_consensus.cc:2229] T f3032262e5544017a94726588d4f6afe P 77f51c177c5f49679dfb01ce597d07c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:29:12.210930 15445 raft_consensus.cc:2258] T f3032262e5544017a94726588d4f6afe P 77f51c177c5f49679dfb01ce597d07c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:12.211409 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 14048
I20260808 14:29:12.213045 15443 tablet_replica.cc:330] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7: stopping tablet replica
I20260808 14:29:12.213382 15443 raft_consensus.cc:2229] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:29:12.213408 15444 ts_tablet_manager.cc:1905] T f3032262e5544017a94726588d4f6afe P 6907de3a9a4144acbce31a6b7f0d6200: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:29:12.214856 15445 ts_tablet_manager.cc:1905] T f3032262e5544017a94726588d4f6afe P 77f51c177c5f49679dfb01ce597d07c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:29:12.218094 15445 ts_tablet_manager.cc:1918] T f3032262e5544017a94726588d4f6afe P 77f51c177c5f49679dfb01ce597d07c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:29:12.218608 15443 raft_consensus.cc:2258] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:12.218943 15445 log.cc:1196] T f3032262e5544017a94726588d4f6afe P 77f51c177c5f49679dfb01ce597d07c8: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-0/wal/wals/f3032262e5544017a94726588d4f6afe
I20260808 14:29:12.219525 15445 ts_tablet_manager.cc:1939] T f3032262e5544017a94726588d4f6afe P 77f51c177c5f49679dfb01ce597d07c8: Deleting consensus metadata
I20260808 14:29:12.220535 15443 ts_tablet_manager.cc:1905] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 14:29:12.222244 14983 connection.cc:535] server connection from 127.0.0.1:37184 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 14:29:12.223954 15443 ts_tablet_manager.cc:1918] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:29:12.224064 15443 log.cc:1196] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/f3032262e5544017a94726588d4f6afe
I20260808 14:29:12.224442 15443 ts_tablet_manager.cc:1939] T f3032262e5544017a94726588d4f6afe P fef7bd05dbce458392c26470cccb0bb7: Deleting consensus metadata
W20260808 14:29:12.225622 15269 connection.cc:441] server connection from 127.0.0.1:48996 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:48996 (request call id 1) could send its response
I20260808 14:29:12.227933 15444 ts_tablet_manager.cc:1918] T f3032262e5544017a94726588d4f6afe P 6907de3a9a4144acbce31a6b7f0d6200: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:29:12.228160 15444 log.cc:1196] T f3032262e5544017a94726588d4f6afe P 6907de3a9a4144acbce31a6b7f0d6200: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/f3032262e5544017a94726588d4f6afe
I20260808 14:29:12.228639 15444 ts_tablet_manager.cc:1939] T f3032262e5544017a94726588d4f6afe P 6907de3a9a4144acbce31a6b7f0d6200: Deleting consensus metadata
W20260808 14:29:12.230065 15128 connection.cc:441] server connection from 127.0.0.1:54002 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54002 (request call id 1) could send its response
I20260808 14:29:12.237555 15400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:12.210079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49694 (local address 127.7.125.125:33639)
0808 14:29:12.210196 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:12.210201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:12.221031 (+ 10830us) server_negotiation.cc:407] Connection header received
0808 14:29:12.221245 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:12.221249 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:12.221302 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:12.221354 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:12.223716 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:12.224574 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:12.225878 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:12.226218 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:12.226314 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:12.232236 (+ 5922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:12.232239 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:12.232242 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:12.232271 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:12.234173 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:12.234693 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:12.234696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:12.234698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:12.234793 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:12.235533 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:12.235538 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:12.235543 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:12.235805 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:12.235938 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:12.237168 (+ 1230us) server_negotiation.cc:299] Negotiation successful
0808 14:29:12.237321 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 14:29:12.239967 15396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:12.227694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57716 (local address 127.7.125.124:38861)
0808 14:29:12.227840 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:12.227845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:12.229861 (+ 2016us) server_negotiation.cc:407] Connection header received
0808 14:29:12.230404 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:12.230411 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:12.230490 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:12.230571 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:12.231729 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:12.232529 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:12.233934 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:12.234316 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:12.234399 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:12.236125 (+ 1726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:12.236129 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:12.236131 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:12.236163 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:12.237957 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:12.238430 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:12.238434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:12.238435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:12.238520 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:12.238843 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:12.238847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:12.238849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:12.239117 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:12.239233 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:12.239631 (+ 398us) server_negotiation.cc:299] Negotiation successful
0808 14:29:12.239765 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 14:29:12.246958 15106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:12.210284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57704 (local address 127.7.125.124:38861)
0808 14:29:12.210393 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:12.210397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:12.236811 (+ 26414us) server_negotiation.cc:407] Connection header received
0808 14:29:12.236859 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:12.236862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:12.236912 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:12.236993 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:12.239698 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:12.240412 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:12.241306 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:12.241602 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:12.241706 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:12.243420 (+ 1714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:12.243424 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:12.243426 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:12.243450 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:12.245101 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:12.245588 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:12.245594 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:12.245598 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:12.245692 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:12.245982 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:12.245988 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:12.245991 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:12.246224 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:12.246376 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:12.246646 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 14:29:12.246765 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 14:29:12.250146 14846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:12.228665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49710 (local address 127.7.125.125:33639)
0808 14:29:12.228794 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:12.228800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:12.230094 (+ 1294us) server_negotiation.cc:407] Connection header received
0808 14:29:12.230648 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:12.230652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:12.230706 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:12.230761 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:12.242968 (+ 12207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:12.243622 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:12.244836 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:12.245201 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:12.245271 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:12.246978 (+ 1707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:12.247005 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:12.247009 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:12.247038 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:12.248564 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:12.248978 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:12.248982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:12.248984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:12.249069 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:12.249336 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:12.249340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:12.249342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:12.249521 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:12.249645 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:12.249873 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 14:29:12.249985 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 14:29:13.195525 15455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:13.194932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.65:34119)
0808 14:29:13.195272 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:13.195417 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":88,"threads_started":1}
W20260808 14:29:13.195921 15103 heartbeater.cc:644] Failed to heartbeat to 127.7.125.126:37133 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
I20260808 14:29:13.197197 15456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:13.196762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.67:48265)
0808 14:29:13.197008 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:13.197094 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":81,"threads_started":1}
W20260808 14:29:13.197733 15386 heartbeater.cc:644] Failed to heartbeat to 127.7.125.126:37133 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
I20260808 14:29:13.200533 15457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:13.200167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.66:59677)
0808 14:29:13.200367 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:13.200459 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":68,"threads_started":1}
W20260808 14:29:13.200807 15244 heartbeater.cc:644] Failed to heartbeat to 127.7.125.126:37133 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
I20260808 14:29:13.707063 15459 raft_consensus.cc:482] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1fbb4311136492580021310c6a777e6)
I20260808 14:29:13.707331 15459 raft_consensus.cc:504] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: false } }
I20260808 14:29:13.707408 15460 raft_consensus.cc:482] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1fbb4311136492580021310c6a777e6)
I20260808 14:29:13.707608 15460 raft_consensus.cc:504] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: false } }
I20260808 14:29:13.708370 15459 leader_election.cc:288] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d1fbb4311136492580021310c6a777e6 (127.7.125.126:37133), a61f7082b8a84de8b08e73fc67bab028 (127.7.125.124:38861)
I20260808 14:29:13.708488 15460 leader_election.cc:288] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d1fbb4311136492580021310c6a777e6 (127.7.125.126:37133), d87c0c9ba2a540778055acacfc93fc35 (127.7.125.125:33639)
I20260808 14:29:13.709127 15462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:13.708646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.0.0.1:56456)
0808 14:29:13.708913 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:13.709000 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":82,"threads_started":1}
I20260808 14:29:13.709182 15463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:13.708800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.0.0.1:56464)
0808 14:29:13.709009 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:13.709084 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":65,"threads_started":1}
W20260808 14:29:13.709398 14699 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
W20260808 14:29:13.709448 14896 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
I20260808 14:29:13.710399 15462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:13.710150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.0.0.1:56476)
0808 14:29:13.710275 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:13.710332 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260808 14:29:13.710500 14699 leader_election.cc:334] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d1fbb4311136492580021310c6a777e6 (127.7.125.126:37133): Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
I20260808 14:29:13.710723 15464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:13.710303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.0.0.1:56482)
0808 14:29:13.710436 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:13.710658 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260808 14:29:13.710827 14896 leader_election.cc:334] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d1fbb4311136492580021310c6a777e6 (127.7.125.126:37133): Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
I20260808 14:29:13.713115 15461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:13.708603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:38861 (local address 127.0.0.1:57732)
0808 14:29:13.708837 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:13.708854 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:13.708961 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:13.709849 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:13.709855 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:13.709879 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:13.710008 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:13.710018 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:13.711123 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:13.711128 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:13.711924 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:13.711933 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:13.712620 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:13.712623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:13.712791 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:13.712811 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 14:29:13.712913 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 14:29:13.712988 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":42,"thread_start_us":74,"threads_started":1}
I20260808 14:29:13.713114 15466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:13.709261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57732 (local address 127.7.125.124:38861)
0808 14:29:13.709597 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:13.709604 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:13.709621 (+ 17us) server_negotiation.cc:407] Connection header received
0808 14:29:13.709671 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:13.709673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:13.709718 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:13.710126 (+ 408us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:13.710230 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:13.711004 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:13.712059 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:13.712497 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:13.712608 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:13.712666 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:13.712931 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 14:29:13.713003 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":65,"threads_started":1}
I20260808 14:29:13.713454 15463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:13.709078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:33639 (local address 127.0.0.1:49714)
0808 14:29:13.709355 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:13.709370 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:13.709460 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:13.709875 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:13.709878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:13.709898 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:13.710002 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:13.710009 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:13.711157 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:13.711159 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:13.711934 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:13.711942 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:13.712983 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:13.712987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:13.713095 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:13.713116 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 14:29:13.713216 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 14:29:13.713314 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":62,"threads_started":1}
I20260808 14:29:13.713569 15465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:13.709174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49714 (local address 127.7.125.125:33639)
0808 14:29:13.709430 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:13.709435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:13.709451 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:29:13.709594 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:13.709597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:13.709651 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:13.709734 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:13.710132 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:13.711010 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:13.712269 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:13.712855 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:13.712981 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:13.713042 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:13.713359 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 14:29:13.713434 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":181,"thread_start_us":68,"threads_started":1}
I20260808 14:29:13.713719 14919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d87c0c9ba2a540778055acacfc93fc35" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a61f7082b8a84de8b08e73fc67bab028" is_pre_election: true
I20260808 14:29:13.713977 14919 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d87c0c9ba2a540778055acacfc93fc35 in term 1.
I20260808 14:29:13.714013 14722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a61f7082b8a84de8b08e73fc67bab028" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d87c0c9ba2a540778055acacfc93fc35" is_pre_election: true
I20260808 14:29:13.714205 14722 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a61f7082b8a84de8b08e73fc67bab028 in term 1.
I20260808 14:29:13.714463 14697 leader_election.cc:302] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a61f7082b8a84de8b08e73fc67bab028, d87c0c9ba2a540778055acacfc93fc35; no voters: d1fbb4311136492580021310c6a777e6
I20260808 14:29:13.714501 14896 leader_election.cc:302] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a61f7082b8a84de8b08e73fc67bab028, d87c0c9ba2a540778055acacfc93fc35; no voters: d1fbb4311136492580021310c6a777e6
I20260808 14:29:13.714704 15459 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 14:29:13.714766 15459 raft_consensus.cc:482] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d1fbb4311136492580021310c6a777e6)
I20260808 14:29:13.714804 15459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 1 FOLLOWER]: Advancing to term 2
I20260808 14:29:13.714934 15460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 14:29:13.715034 15460 raft_consensus.cc:482] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d1fbb4311136492580021310c6a777e6)
I20260808 14:29:13.715092 15460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 1 FOLLOWER]: Advancing to term 2
I20260808 14:29:13.716056 15459 raft_consensus.cc:504] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: false } }
I20260808 14:29:13.716332 15460 raft_consensus.cc:504] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: false } }
I20260808 14:29:13.716492 15459 leader_election.cc:288] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [CANDIDATE]: Term 2 election: Requested vote from peers d1fbb4311136492580021310c6a777e6 (127.7.125.126:37133), a61f7082b8a84de8b08e73fc67bab028 (127.7.125.124:38861)
I20260808 14:29:13.716773 15460 leader_election.cc:288] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [CANDIDATE]: Term 2 election: Requested vote from peers d1fbb4311136492580021310c6a777e6 (127.7.125.126:37133), d87c0c9ba2a540778055acacfc93fc35 (127.7.125.125:33639)
I20260808 14:29:13.716853 15461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:13.716612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.0.0.1:56502)
0808 14:29:13.716728 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:13.716790 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 14:29:13.717185 14919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d87c0c9ba2a540778055acacfc93fc35" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a61f7082b8a84de8b08e73fc67bab028"
I20260808 14:29:13.717202 14722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a61f7082b8a84de8b08e73fc67bab028" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d87c0c9ba2a540778055acacfc93fc35"
I20260808 14:29:13.717374 14722 raft_consensus.cc:2379] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a61f7082b8a84de8b08e73fc67bab028 in current term 2: Already voted for candidate d87c0c9ba2a540778055acacfc93fc35 in this term.
I20260808 14:29:13.717358 15463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:13.716848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.0.0.1:56506)
0808 14:29:13.717229 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:13.717289 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305}
I20260808 14:29:13.717361 14919 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d87c0c9ba2a540778055acacfc93fc35 in current term 2: Already voted for candidate a61f7082b8a84de8b08e73fc67bab028 in this term.
I20260808 14:29:13.718101 15461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:13.717880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.0.0.1:56516)
0808 14:29:13.717982 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:13.718030 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 14:29:13.718302 15463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:13.718058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.0.0.1:56522)
0808 14:29:13.718178 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:13.718235 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 14:29:13.718292 14699 leader_election.cc:334] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d1fbb4311136492580021310c6a777e6 (127.7.125.126:37133): Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
I20260808 14:29:13.718379 14699 leader_election.cc:302] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d87c0c9ba2a540778055acacfc93fc35; no voters: a61f7082b8a84de8b08e73fc67bab028, d1fbb4311136492580021310c6a777e6
W20260808 14:29:13.718425 14896 leader_election.cc:334] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d1fbb4311136492580021310c6a777e6 (127.7.125.126:37133): Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
I20260808 14:29:13.718513 14896 leader_election.cc:302] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a61f7082b8a84de8b08e73fc67bab028; no voters: d1fbb4311136492580021310c6a777e6, d87c0c9ba2a540778055acacfc93fc35
I20260808 14:29:13.718511 15459 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 14:29:13.718690 15460 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 14:29:14.197129 15469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:14.196603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.65:40811)
0808 14:29:14.196932 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:14.197007 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":101,"threads_started":1}
I20260808 14:29:14.198722 15470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:14.198304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.67:51141)
0808 14:29:14.198551 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:14.198608 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":86,"threads_started":1}
I20260808 14:29:14.201617 15471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:14.201282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.66:35921)
0808 14:29:14.201494 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:14.201544 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":68,"threads_started":1}
I20260808 14:29:15.198585 15472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.198060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.65:52071)
0808 14:29:15.198334 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.198448 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":100,"threads_started":1}
I20260808 14:29:15.199813 15473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.199376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.67:39343)
0808 14:29:15.199642 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.199710 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":95,"threads_started":1}
I20260808 14:29:15.202644 15474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.202251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.66:52463)
0808 14:29:15.202502 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.202570 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":73,"threads_started":1}
I20260808 14:29:15.588194 15475 raft_consensus.cc:482] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:29:15.588346 15475 raft_consensus.cc:504] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: false } }
I20260808 14:29:15.588747 15475 leader_election.cc:288] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d1fbb4311136492580021310c6a777e6 (127.7.125.126:37133), a61f7082b8a84de8b08e73fc67bab028 (127.7.125.124:38861)
I20260808 14:29:15.589252 14919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d87c0c9ba2a540778055acacfc93fc35" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a61f7082b8a84de8b08e73fc67bab028" is_pre_election: true
I20260808 14:29:15.589375 15476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.588985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.0.0.1:56758)
0808 14:29:15.589204 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.589264 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":72,"threads_started":1}
I20260808 14:29:15.589391 14919 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d87c0c9ba2a540778055acacfc93fc35 in term 2.
I20260808 14:29:15.589751 14697 leader_election.cc:302] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a61f7082b8a84de8b08e73fc67bab028, d87c0c9ba2a540778055acacfc93fc35; no voters:
I20260808 14:29:15.589942 15475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260808 14:29:15.590001 15475 raft_consensus.cc:482] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:29:15.590029 15475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 2 FOLLOWER]: Advancing to term 3
I20260808 14:29:15.590303 15476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.590059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.0.0.1:56774)
0808 14:29:15.590191 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.590246 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 14:29:15.590430 14699 leader_election.cc:334] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1fbb4311136492580021310c6a777e6 (127.7.125.126:37133): Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
I20260808 14:29:15.591256 15475 raft_consensus.cc:504] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: false } }
I20260808 14:29:15.591595 15475 leader_election.cc:288] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [CANDIDATE]: Term 3 election: Requested vote from peers d1fbb4311136492580021310c6a777e6 (127.7.125.126:37133), a61f7082b8a84de8b08e73fc67bab028 (127.7.125.124:38861)
I20260808 14:29:15.591964 15476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.591747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.0.0.1:56784)
0808 14:29:15.591861 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.591912 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 14:29:15.591969 14919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d87c0c9ba2a540778055acacfc93fc35" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a61f7082b8a84de8b08e73fc67bab028"
I20260808 14:29:15.592089 14919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 2 FOLLOWER]: Advancing to term 3
I20260808 14:29:15.592649 15476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.592452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.0.0.1:56792)
0808 14:29:15.592549 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.592603 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 14:29:15.592770 14699 leader_election.cc:334] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d1fbb4311136492580021310c6a777e6 (127.7.125.126:37133): Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
I20260808 14:29:15.593173 14919 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d87c0c9ba2a540778055acacfc93fc35 in term 3.
I20260808 14:29:15.593457 14697 leader_election.cc:302] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a61f7082b8a84de8b08e73fc67bab028, d87c0c9ba2a540778055acacfc93fc35; no voters: d1fbb4311136492580021310c6a777e6
I20260808 14:29:15.593611 15475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 14:29:15.593801 15475 raft_consensus.cc:686] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [term 3 LEADER]: Becoming Leader. State: Replica: d87c0c9ba2a540778055acacfc93fc35, State: Running, Role: LEADER
I20260808 14:29:15.594065 15475 consensus_queue.cc:237] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: false } }
I20260808 14:29:15.595079 15478 sys_catalog.cc:455] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d87c0c9ba2a540778055acacfc93fc35. Latest consensus state: current_term: 3 leader_uuid: "d87c0c9ba2a540778055acacfc93fc35" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: false } } }
I20260808 14:29:15.595221 15478 sys_catalog.cc:458] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:15.595557 15480 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:29:15.598446 15480 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:29:15.598901 15480 catalog_manager.cc:1259] Loaded cluster ID: e6e375437ebd404ab1a6a3c7859fe228
I20260808 14:29:15.598973 15480 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:29:15.599578 15480 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:29:15.599990 15480 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35: Loaded TSK: 0
I20260808 14:29:15.600692 15480 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:29:15.641767 14711 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:29:15.645004 14711 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:29:15.648386 15476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.648099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.0.0.1:56810)
0808 14:29:15.648236 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.648309 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 14:29:15.648499 14919 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [term 3 FOLLOWER]: Refusing update from remote peer d87c0c9ba2a540778055acacfc93fc35: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260808 14:29:15.649042 15475 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 14:29:15.649274 15476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.648980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.0.0.1:56816)
0808 14:29:15.649096 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.649149 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 14:29:15.649403 14699 consensus_peers.cc:487] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 -> Peer d1fbb4311136492580021310c6a777e6 (127.7.125.126:37133): Couldn't send request to peer d1fbb4311136492580021310c6a777e6. Status: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 14:29:15.650537 15482 sys_catalog.cc:455] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d87c0c9ba2a540778055acacfc93fc35. Latest consensus state: current_term: 3 leader_uuid: "d87c0c9ba2a540778055acacfc93fc35" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: false } } }
I20260808 14:29:15.650691 15482 sys_catalog.cc:458] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:29:15.651842 15475 sys_catalog.cc:455] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "d87c0c9ba2a540778055acacfc93fc35" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: false } } }
I20260808 14:29:15.651970 15475 sys_catalog.cc:458] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35 [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:15.652231 15482 sys_catalog.cc:455] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "d87c0c9ba2a540778055acacfc93fc35" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1fbb4311136492580021310c6a777e6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 37133 } } peers { permanent_uuid: "d87c0c9ba2a540778055acacfc93fc35" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 33639 } attrs { promote: false } } peers { permanent_uuid: "a61f7082b8a84de8b08e73fc67bab028" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38861 } attrs { promote: false } } }
I20260808 14:29:15.652369 15482 sys_catalog.cc:458] T 00000000000000000000000000000000 P a61f7082b8a84de8b08e73fc67bab028 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:29:15.663286 15476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.657726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:37109 (local address 127.0.0.1:49020)
0808 14:29:15.657925 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.657941 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:15.658170 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:15.659575 (+ 1405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:15.659581 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:15.659598 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:15.659699 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:15.659706 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:15.660692 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:15.660697 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:15.661704 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:15.661714 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:15.662727 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:15.662731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:15.662856 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:15.662890 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 14:29:15.662967 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 14:29:15.663098 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":37}
I20260808 14:29:15.663439 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.658533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54008 (local address 127.7.125.66:44269)
0808 14:29:15.658967 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:15.659020 (+ 53us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:15.659039 (+ 19us) server_negotiation.cc:407] Connection header received
0808 14:29:15.659095 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:15.659098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:15.659150 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:15.659546 (+ 396us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:15.659671 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:15.660515 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:15.662049 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:15.662749 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:15.662903 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:15.663024 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:15.663188 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 14:29:15.663268 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":103,"threads_started":1}
I20260808 14:29:15.663446 15489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.658303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:44269 (local address 127.0.0.1:54008)
0808 14:29:15.658683 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.658699 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:15.658789 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:15.659284 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:15.659287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:15.659302 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:15.659425 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:15.659431 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:15.660630 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:15.660633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:15.661893 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:15.661904 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:15.662928 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:15.662934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:15.663046 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:15.663069 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 14:29:15.663149 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 14:29:15.663218 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":83,"thread_start_us":108,"threads_started":1}
I20260808 14:29:15.664026 15488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.658303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:35815 (local address 127.0.0.1:37190)
0808 14:29:15.658589 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.658606 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:15.658703 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:15.659380 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:15.659383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:15.659405 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:15.659522 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:15.659531 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:15.660502 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:15.660508 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:15.662402 (+ 1894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:15.662412 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:15.663510 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:15.663515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:15.663724 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:15.663749 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 14:29:15.663830 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 14:29:15.663903 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":36,"thread_start_us":130,"threads_started":1}
I20260808 14:29:15.664028 15174 tablet_service.cc:1467] Processing CreateTablet for tablet 651ee33b141a42ef94360e2fa0442f0d (DEFAULT_TABLE table=test-table [id=6a67db2fe2574bf9b4f92a3bd2e11f0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:15.664115 15492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.658763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37190 (local address 127.7.125.65:35815)
0808 14:29:15.659079 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:15.659085 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:15.659103 (+ 18us) server_negotiation.cc:407] Connection header received
0808 14:29:15.659161 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:15.659175 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:15.659240 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:15.659662 (+ 422us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:15.659810 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:15.660369 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:15.662529 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:15.663250 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:15.663355 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:15.663416 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:15.663875 (+ 459us) server_negotiation.cc:299] Negotiation successful
0808 14:29:15.663969 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":107,"threads_started":1}
I20260808 14:29:15.664367 15174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 651ee33b141a42ef94360e2fa0442f0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:15.664763 15487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.657863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49020 (local address 127.7.125.67:37109)
0808 14:29:15.659313 (+ 1450us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:15.659318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:15.659332 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:29:15.659413 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:15.659415 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:15.659470 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:15.659560 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:15.659822 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:15.660598 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:15.661856 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:15.662585 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:15.662694 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:15.662814 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:15.664519 (+ 1705us) server_negotiation.cc:299] Negotiation successful
0808 14:29:15.664621 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1366,"thread_start_us":109,"threads_started":1}
I20260808 14:29:15.664897 15032 tablet_service.cc:1467] Processing CreateTablet for tablet 651ee33b141a42ef94360e2fa0442f0d (DEFAULT_TABLE table=test-table [id=6a67db2fe2574bf9b4f92a3bd2e11f0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:15.665184 15032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 651ee33b141a42ef94360e2fa0442f0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:15.665472 15315 tablet_service.cc:1467] Processing CreateTablet for tablet 651ee33b141a42ef94360e2fa0442f0d (DEFAULT_TABLE table=test-table [id=6a67db2fe2574bf9b4f92a3bd2e11f0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:15.665772 15315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 651ee33b141a42ef94360e2fa0442f0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:15.667240 15493 tablet_bootstrap.cc:492] T 651ee33b141a42ef94360e2fa0442f0d P 6907de3a9a4144acbce31a6b7f0d6200: Bootstrap starting.
I20260808 14:29:15.667912 15494 tablet_bootstrap.cc:492] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8: Bootstrap starting.
I20260808 14:29:15.668325 15493 tablet_bootstrap.cc:654] T 651ee33b141a42ef94360e2fa0442f0d P 6907de3a9a4144acbce31a6b7f0d6200: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:15.668955 15494 tablet_bootstrap.cc:654] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:15.669003 15495 tablet_bootstrap.cc:492] T 651ee33b141a42ef94360e2fa0442f0d P fef7bd05dbce458392c26470cccb0bb7: Bootstrap starting.
I20260808 14:29:15.669849 15495 tablet_bootstrap.cc:654] T 651ee33b141a42ef94360e2fa0442f0d P fef7bd05dbce458392c26470cccb0bb7: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:15.670198 15494 tablet_bootstrap.cc:492] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8: No bootstrap required, opened a new log
I20260808 14:29:15.670305 15494 ts_tablet_manager.cc:1397] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 14:29:15.670912 15494 raft_consensus.cc:348] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fef7bd05dbce458392c26470cccb0bb7" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37109 } } peers { permanent_uuid: "77f51c177c5f49679dfb01ce597d07c8" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35815 } } peers { permanent_uuid: "6907de3a9a4144acbce31a6b7f0d6200" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44269 } }
I20260808 14:29:15.671056 15494 raft_consensus.cc:374] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:15.671125 15494 raft_consensus.cc:729] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77f51c177c5f49679dfb01ce597d07c8, State: Initialized, Role: FOLLOWER
I20260808 14:29:15.671183 15495 tablet_bootstrap.cc:492] T 651ee33b141a42ef94360e2fa0442f0d P fef7bd05dbce458392c26470cccb0bb7: No bootstrap required, opened a new log
I20260808 14:29:15.671242 15493 tablet_bootstrap.cc:492] T 651ee33b141a42ef94360e2fa0442f0d P 6907de3a9a4144acbce31a6b7f0d6200: No bootstrap required, opened a new log
I20260808 14:29:15.671271 15495 ts_tablet_manager.cc:1397] T 651ee33b141a42ef94360e2fa0442f0d P fef7bd05dbce458392c26470cccb0bb7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 14:29:15.671322 15493 ts_tablet_manager.cc:1397] T 651ee33b141a42ef94360e2fa0442f0d P 6907de3a9a4144acbce31a6b7f0d6200: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 14:29:15.671306 15494 consensus_queue.cc:260] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fef7bd05dbce458392c26470cccb0bb7" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37109 } } peers { permanent_uuid: "77f51c177c5f49679dfb01ce597d07c8" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35815 } } peers { permanent_uuid: "6907de3a9a4144acbce31a6b7f0d6200" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44269 } }
I20260808 14:29:15.671723 15494 ts_tablet_manager.cc:1428] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:29:15.671734 15495 raft_consensus.cc:348] T 651ee33b141a42ef94360e2fa0442f0d P fef7bd05dbce458392c26470cccb0bb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fef7bd05dbce458392c26470cccb0bb7" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37109 } } peers { permanent_uuid: "77f51c177c5f49679dfb01ce597d07c8" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35815 } } peers { permanent_uuid: "6907de3a9a4144acbce31a6b7f0d6200" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44269 } }
I20260808 14:29:15.671852 15495 raft_consensus.cc:374] T 651ee33b141a42ef94360e2fa0442f0d P fef7bd05dbce458392c26470cccb0bb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:15.671821 15493 raft_consensus.cc:348] T 651ee33b141a42ef94360e2fa0442f0d P 6907de3a9a4144acbce31a6b7f0d6200 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fef7bd05dbce458392c26470cccb0bb7" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37109 } } peers { permanent_uuid: "77f51c177c5f49679dfb01ce597d07c8" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35815 } } peers { permanent_uuid: "6907de3a9a4144acbce31a6b7f0d6200" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44269 } }
I20260808 14:29:15.671890 15495 raft_consensus.cc:729] T 651ee33b141a42ef94360e2fa0442f0d P fef7bd05dbce458392c26470cccb0bb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef7bd05dbce458392c26470cccb0bb7, State: Initialized, Role: FOLLOWER
I20260808 14:29:15.671931 15493 raft_consensus.cc:374] T 651ee33b141a42ef94360e2fa0442f0d P 6907de3a9a4144acbce31a6b7f0d6200 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:15.671954 15493 raft_consensus.cc:729] T 651ee33b141a42ef94360e2fa0442f0d P 6907de3a9a4144acbce31a6b7f0d6200 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6907de3a9a4144acbce31a6b7f0d6200, State: Initialized, Role: FOLLOWER
I20260808 14:29:15.672021 15495 consensus_queue.cc:260] T 651ee33b141a42ef94360e2fa0442f0d P fef7bd05dbce458392c26470cccb0bb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fef7bd05dbce458392c26470cccb0bb7" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37109 } } peers { permanent_uuid: "77f51c177c5f49679dfb01ce597d07c8" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35815 } } peers { permanent_uuid: "6907de3a9a4144acbce31a6b7f0d6200" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44269 } }
I20260808 14:29:15.672282 15493 consensus_queue.cc:260] T 651ee33b141a42ef94360e2fa0442f0d P 6907de3a9a4144acbce31a6b7f0d6200 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fef7bd05dbce458392c26470cccb0bb7" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37109 } } peers { permanent_uuid: "77f51c177c5f49679dfb01ce597d07c8" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35815 } } peers { permanent_uuid: "6907de3a9a4144acbce31a6b7f0d6200" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44269 } }
I20260808 14:29:15.672438 15495 ts_tablet_manager.cc:1428] T 651ee33b141a42ef94360e2fa0442f0d P fef7bd05dbce458392c26470cccb0bb7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:29:15.673291 15473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.672996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.67:48377)
0808 14:29:15.673135 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.673214 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 14:29:15.673779 15472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.672894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.65:44999)
0808 14:29:15.673633 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.673705 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":661}
I20260808 14:29:15.675232 15493 ts_tablet_manager.cc:1428] T 651ee33b141a42ef94360e2fa0442f0d P 6907de3a9a4144acbce31a6b7f0d6200: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260808 14:29:15.676086 15474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.675719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.66:35983)
0808 14:29:15.675950 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.676012 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260808 14:29:15.676525 15387 heartbeater.cc:503] Master 127.7.125.125:33639 was elected leader, sending a full tablet report...
I20260808 14:29:15.676918 15104 heartbeater.cc:503] Master 127.7.125.125:33639 was elected leader, sending a full tablet report...
I20260808 14:29:15.676944 15473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.676651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.67:59785)
0808 14:29:15.676815 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.676871 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260808 14:29:15.677479 15472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.676651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.65:42535)
0808 14:29:15.677346 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.677407 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":620}
I20260808 14:29:15.678642 15245 heartbeater.cc:503] Master 127.7.125.125:33639 was elected leader, sending a full tablet report...
I20260808 14:29:15.678982 15474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.678757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.66:50429)
0808 14:29:15.678848 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.678897 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 14:29:15.701795 15499 raft_consensus.cc:482] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:29:15.701953 15499 raft_consensus.cc:504] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fef7bd05dbce458392c26470cccb0bb7" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37109 } } peers { permanent_uuid: "77f51c177c5f49679dfb01ce597d07c8" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35815 } } peers { permanent_uuid: "6907de3a9a4144acbce31a6b7f0d6200" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44269 } }
I20260808 14:29:15.703145 15499 leader_election.cc:288] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fef7bd05dbce458392c26470cccb0bb7 (127.7.125.67:37109), 6907de3a9a4144acbce31a6b7f0d6200 (127.7.125.66:44269)
I20260808 14:29:15.708151 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.703500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:57609 (local address 127.7.125.66:44269)
0808 14:29:15.703630 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:15.703636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:15.703656 (+ 20us) server_negotiation.cc:407] Connection header received
0808 14:29:15.703782 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:15.703785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:15.703838 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:15.703935 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:15.704250 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:15.705038 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:15.706759 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:15.707397 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:15.707523 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:15.707575 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:15.707949 (+ 374us) server_negotiation.cc:299] Negotiation successful
0808 14:29:15.708019 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 14:29:15.708151 15487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.703530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:55703 (local address 127.7.125.67:37109)
0808 14:29:15.703653 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:15.703657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:15.704039 (+ 382us) server_negotiation.cc:407] Connection header received
0808 14:29:15.704126 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:15.704128 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:15.704168 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:15.704258 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:15.704497 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:15.705198 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:15.706759 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:15.707376 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:15.707522 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:15.707575 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:15.707941 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 14:29:15.708019 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 14:29:15.708232 15472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.703386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:44269 (local address 127.7.125.65:57609)
0808 14:29:15.703533 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.703549 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:15.703659 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:15.703952 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:15.703957 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:15.703984 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:15.704133 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:15.704140 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:15.705143 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:15.705145 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:15.706632 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:15.706644 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:15.707541 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:15.707545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:15.707778 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:15.707838 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 14:29:15.707906 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 14:29:15.708064 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":200}
I20260808 14:29:15.708348 15502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.703386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:37109 (local address 127.7.125.65:55703)
0808 14:29:15.703928 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.703940 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:15.704044 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:15.704281 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:15.704283 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:15.704297 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:15.704387 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:15.704391 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:15.705292 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:15.705295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:15.706635 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:15.706644 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:15.707541 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:15.707545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:15.707776 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:15.707823 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 14:29:15.707906 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 14:29:15.708064 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":332,"thread_start_us":143,"threads_started":1}
I20260808 14:29:15.708758 15196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "651ee33b141a42ef94360e2fa0442f0d" candidate_uuid: "77f51c177c5f49679dfb01ce597d07c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6907de3a9a4144acbce31a6b7f0d6200" is_pre_election: true
I20260808 14:29:15.708956 15196 raft_consensus.cc:2454] T 651ee33b141a42ef94360e2fa0442f0d P 6907de3a9a4144acbce31a6b7f0d6200 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77f51c177c5f49679dfb01ce597d07c8 in term 0.
I20260808 14:29:15.709019 15337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "651ee33b141a42ef94360e2fa0442f0d" candidate_uuid: "77f51c177c5f49679dfb01ce597d07c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef7bd05dbce458392c26470cccb0bb7" is_pre_election: true
I20260808 14:29:15.709265 15337 raft_consensus.cc:2454] T 651ee33b141a42ef94360e2fa0442f0d P fef7bd05dbce458392c26470cccb0bb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77f51c177c5f49679dfb01ce597d07c8 in term 0.
I20260808 14:29:15.709411 14985 leader_election.cc:302] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6907de3a9a4144acbce31a6b7f0d6200, 77f51c177c5f49679dfb01ce597d07c8; no voters:
I20260808 14:29:15.709729 15499 raft_consensus.cc:2781] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:29:15.709789 15499 raft_consensus.cc:482] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:29:15.709831 15499 raft_consensus.cc:3037] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:15.710886 15499 raft_consensus.cc:504] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fef7bd05dbce458392c26470cccb0bb7" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37109 } } peers { permanent_uuid: "77f51c177c5f49679dfb01ce597d07c8" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35815 } } peers { permanent_uuid: "6907de3a9a4144acbce31a6b7f0d6200" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44269 } }
I20260808 14:29:15.711314 15499 leader_election.cc:288] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8 [CANDIDATE]: Term 1 election: Requested vote from peers fef7bd05dbce458392c26470cccb0bb7 (127.7.125.67:37109), 6907de3a9a4144acbce31a6b7f0d6200 (127.7.125.66:44269)
I20260808 14:29:15.711598 15337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "651ee33b141a42ef94360e2fa0442f0d" candidate_uuid: "77f51c177c5f49679dfb01ce597d07c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fef7bd05dbce458392c26470cccb0bb7"
I20260808 14:29:15.711735 15337 raft_consensus.cc:3037] T 651ee33b141a42ef94360e2fa0442f0d P fef7bd05dbce458392c26470cccb0bb7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:15.711694 15196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "651ee33b141a42ef94360e2fa0442f0d" candidate_uuid: "77f51c177c5f49679dfb01ce597d07c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6907de3a9a4144acbce31a6b7f0d6200"
I20260808 14:29:15.711846 15196 raft_consensus.cc:3037] T 651ee33b141a42ef94360e2fa0442f0d P 6907de3a9a4144acbce31a6b7f0d6200 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:15.712734 15337 raft_consensus.cc:2454] T 651ee33b141a42ef94360e2fa0442f0d P fef7bd05dbce458392c26470cccb0bb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77f51c177c5f49679dfb01ce597d07c8 in term 1.
I20260808 14:29:15.712746 15196 raft_consensus.cc:2454] T 651ee33b141a42ef94360e2fa0442f0d P 6907de3a9a4144acbce31a6b7f0d6200 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77f51c177c5f49679dfb01ce597d07c8 in term 1.
I20260808 14:29:15.713027 14984 leader_election.cc:302] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77f51c177c5f49679dfb01ce597d07c8, fef7bd05dbce458392c26470cccb0bb7; no voters:
I20260808 14:29:15.713210 15499 raft_consensus.cc:2781] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:29:15.713423 15499 raft_consensus.cc:686] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8 [term 1 LEADER]: Becoming Leader. State: Replica: 77f51c177c5f49679dfb01ce597d07c8, State: Running, Role: LEADER
I20260808 14:29:15.713693 15499 consensus_queue.cc:237] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fef7bd05dbce458392c26470cccb0bb7" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37109 } } peers { permanent_uuid: "77f51c177c5f49679dfb01ce597d07c8" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35815 } } peers { permanent_uuid: "6907de3a9a4144acbce31a6b7f0d6200" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44269 } }
I20260808 14:29:15.714089 15502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.713889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.65:57089)
0808 14:29:15.713978 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.714027 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 14:29:15.716229 14711 catalog_manager.cc:5495] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 77f51c177c5f49679dfb01ce597d07c8 (127.7.125.65). New cstate: current_term: 1 leader_uuid: "77f51c177c5f49679dfb01ce597d07c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fef7bd05dbce458392c26470cccb0bb7" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77f51c177c5f49679dfb01ce597d07c8" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6907de3a9a4144acbce31a6b7f0d6200" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44269 } health_report { overall_health: UNKNOWN } } }
I20260808 14:29:15.730032 15492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.726592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32864 (local address 127.7.125.65:35815)
0808 14:29:15.726728 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:15.726733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:15.726749 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:29:15.726845 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:15.726847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:15.726900 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:15.727039 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:29:15.727268 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:15.727795 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:15.728639 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:15.728857 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:15.728960 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:15.729225 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:29:15.729664 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:29:15.729800 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:15.729837 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 14:29:15.729893 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:29:15.732923 15196 raft_consensus.cc:1261] T 651ee33b141a42ef94360e2fa0442f0d P 6907de3a9a4144acbce31a6b7f0d6200 [term 1 FOLLOWER]: Refusing update from remote peer 77f51c177c5f49679dfb01ce597d07c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:29:15.733116 15337 raft_consensus.cc:1261] T 651ee33b141a42ef94360e2fa0442f0d P fef7bd05dbce458392c26470cccb0bb7 [term 1 FOLLOWER]: Refusing update from remote peer 77f51c177c5f49679dfb01ce597d07c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:29:15.733578 15504 consensus_queue.cc:1035] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6907de3a9a4144acbce31a6b7f0d6200" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 44269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:29:15.733784 15499 consensus_queue.cc:1035] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fef7bd05dbce458392c26470cccb0bb7" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:29:15.736063 15473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.735750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.67:33923)
0808 14:29:15.735921 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.735986 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260808 14:29:15.738545 15502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.738300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.65:34613)
0808 14:29:15.738407 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.738470 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 14:29:15.740545 15473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.740313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.67:34591)
0808 14:29:15.740411 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.740465 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 14:29:15.740859 15474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.735753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.66:56731)
0808 14:29:15.740708 (+ 4955us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.740764 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4864}
I20260808 14:29:15.742249 15474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.741893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.66:41807)
0808 14:29:15.742115 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.742180 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
I20260808 14:29:15.742781 15502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:15.742220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:37133 (local address 127.7.125.65:55259)
0808 14:29:15.742657 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.742711 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366}
I20260808 14:29:15.747438 14712 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:29:15.747602 14712 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:29:15.751468 14712 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d87c0c9ba2a540778055acacfc93fc35: Sending DeleteTablet for 3 replicas of tablet 651ee33b141a42ef94360e2fa0442f0d
I20260808 14:29:15.752606 15032 tablet_service.cc:1514] Processing DeleteTablet for tablet 651ee33b141a42ef94360e2fa0442f0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:15 UTC) from {username='oryx', principal='oryx/127.7.125.125@KRBTEST.COM'} at 127.0.0.1:37190
I20260808 14:29:15.752642 15174 tablet_service.cc:1514] Processing DeleteTablet for tablet 651ee33b141a42ef94360e2fa0442f0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:15 UTC) from {username='oryx', principal='oryx/127.7.125.125@KRBTEST.COM'} at 127.0.0.1:54008
I20260808 14:29:15.752976 15519 tablet_replica.cc:330] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8: stopping tablet replica
I20260808 14:29:15.752995 15315 tablet_service.cc:1514] Processing DeleteTablet for tablet 651ee33b141a42ef94360e2fa0442f0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:15 UTC) from {username='oryx', principal='oryx/127.7.125.125@KRBTEST.COM'} at 127.0.0.1:49020
I20260808 14:29:15.753127 15519 raft_consensus.cc:2229] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:29:15.753365 15521 tablet_replica.cc:330] T 651ee33b141a42ef94360e2fa0442f0d P fef7bd05dbce458392c26470cccb0bb7: stopping tablet replica
I20260808 14:29:15.753479 15519 raft_consensus.cc:2258] T 651ee33b141a42ef94360e2fa0442f0d P 77f51c177c5f49679dfb01ce597d07c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:15.753552 15521 raft_consensus.cc:2229] T 651ee33b141a42ef94360e2fa0442f0d P fef7bd05dbce458392c26470cccb0bb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:29:15.753708 15521 raft_consensus.cc:2258] T 651ee33b141a42ef94360e2fa0442f0d P fef7bd05dbce458392c26470cccb0bb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:15.753702 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 14868
I20260808 14:29:15.756420 15520 tablet_replica.cc:330] T 651ee33b141a42ef94360e2fa0442f0d P 6907de3a9a4144acbce31a6b7f0d6200: stopping tablet replica
I20260808 14:29:15.756562 15520 raft_consensus.cc:2229] T 651ee33b141a42ef94360e2fa0442f0d P 6907de3a9a4144acbce31a6b7f0d6200 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:29:15.756692 15520 raft_consensus.cc:2258] T 651ee33b141a42ef94360e2fa0442f0d P 6907de3a9a4144acbce31a6b7f0d6200 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:15.758244 15520 ts_tablet_manager.cc:1905] T 651ee33b141a42ef94360e2fa0442f0d P 6907de3a9a4144acbce31a6b7f0d6200: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:29:15.762106 15520 ts_tablet_manager.cc:1918] T 651ee33b141a42ef94360e2fa0442f0d P 6907de3a9a4144acbce31a6b7f0d6200: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:29:15.762274 15520 log.cc:1196] T 651ee33b141a42ef94360e2fa0442f0d P 6907de3a9a4144acbce31a6b7f0d6200: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/651ee33b141a42ef94360e2fa0442f0d
I20260808 14:29:15.762602 15520 ts_tablet_manager.cc:1939] T 651ee33b141a42ef94360e2fa0442f0d P 6907de3a9a4144acbce31a6b7f0d6200: Deleting consensus metadata
I20260808 14:29:15.763499 14698 catalog_manager.cc:4841] TS 6907de3a9a4144acbce31a6b7f0d6200 (127.7.125.66:44269): tablet 651ee33b141a42ef94360e2fa0442f0d (table test-table [id=6a67db2fe2574bf9b4f92a3bd2e11f0f]) successfully deleted
W20260808 14:29:15.765476 15488 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:15.765112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:35815 (local address 127.0.0.1:32874)
0808 14:29:15.765269 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:15.765336 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.65:35815: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 14:29:15.765551 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 15110
W20260808 14:29:15.765642 14696 catalog_manager.cc:4584] TS 77f51c177c5f49679dfb01ce597d07c8 (127.7.125.65:35815): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 651ee33b141a42ef94360e2fa0442f0d: Network error: Client connection negotiation failed: client connection to 127.7.125.65:35815: connect: Connection reset by peer (error 104)
I20260808 14:29:15.766952 15521 ts_tablet_manager.cc:1905] T 651ee33b141a42ef94360e2fa0442f0d P fef7bd05dbce458392c26470cccb0bb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:29:15.769506 15521 ts_tablet_manager.cc:1918] T 651ee33b141a42ef94360e2fa0442f0d P fef7bd05dbce458392c26470cccb0bb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:29:15.769586 15521 log.cc:1196] T 651ee33b141a42ef94360e2fa0442f0d P fef7bd05dbce458392c26470cccb0bb7: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/651ee33b141a42ef94360e2fa0442f0d
I20260808 14:29:15.769877 15521 ts_tablet_manager.cc:1939] T 651ee33b141a42ef94360e2fa0442f0d P fef7bd05dbce458392c26470cccb0bb7: Deleting consensus metadata
I20260808 14:29:15.770848 14697 catalog_manager.cc:4841] TS fef7bd05dbce458392c26470cccb0bb7 (127.7.125.67:37109): tablet 651ee33b141a42ef94360e2fa0442f0d (table test-table [id=6a67db2fe2574bf9b4f92a3bd2e11f0f]) successfully deleted
I20260808 14:29:15.772991 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 15250
I20260808 14:29:15.781092 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 14571
I20260808 14:29:15.788033 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 14683
2026-08-08T14:29:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7466 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:29:15 dist-test-slave-w206 krb5kdc[15524](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:29:15 dist-test-slave-w206 krb5kdc[15524](info): set up 2 sockets
Aug 08 14:29:15 dist-test-slave-w206 krb5kdc[15524](info): commencing operation
krb5kdc: starting...
W20260808 14:29:16.864351 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:29:16 dist-test-slave-w206 krb5kdc[15524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:29:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:29:16Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.126.keytab.
Entry for principal oryx/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.126.keytab.
I20260808 14:29:16.994442 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:44075
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.126.keytab
--principal=oryx/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:43391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:44075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:17.109454 15537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:17.109741 15537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:17.109783 15537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:17.113730 15537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:29:17.113793 15537 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 14:29:17.113816 15537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:17.113837 15537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:17.113852 15537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:29:17.113873 15537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:29:17.119204 15537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:43391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:44075
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.126.keytab
--principal=oryx/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:44075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:17.120282 15537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:17.121341 15537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:17.130522 15545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:17.130641 15542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:17.130525 15543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:17.130805 15537 server_base.cc:1034] running on GCE node
I20260808 14:29:17.131423 15537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:17.132627 15537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:17.133826 15537 hybrid_clock.cc:648] HybridClock initialized: now 1786199357133803 us; error 38 us; skew 500 ppm
Aug 08 14:29:17 dist-test-slave-w206 krb5kdc[15524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199357, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:17.136798 15537 init.cc:377] Logged in from keytab as oryx/127.7.125.126@KRBTEST.COM (short username oryx)
I20260808 14:29:17.137792 15537 webserver.cc:460] Webserver started at http://127.7.125.126:37801/ using document root <none> and password file <none>
I20260808 14:29:17.138377 15537 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:17.138473 15537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:17.138701 15537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:17.141063 15537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "dabd75c6d2d442619e1a65914cec0ebe"
format_stamp: "Formatted at 2026-08-08 14:29:17 on dist-test-slave-w206"
server_key: "1202039672fca61b77fa77a7fcfd93ae"
server_key_iv: "b61d4082b5fbc709511bbbbe946ff43a"
server_key_version: "encryptionkey@0"
I20260808 14:29:17.141657 15537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "dabd75c6d2d442619e1a65914cec0ebe"
format_stamp: "Formatted at 2026-08-08 14:29:17 on dist-test-slave-w206"
server_key: "1202039672fca61b77fa77a7fcfd93ae"
server_key_iv: "b61d4082b5fbc709511bbbbe946ff43a"
server_key_version: "encryptionkey@0"
I20260808 14:29:17.145587 15537 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:29:17.148115 15552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:17.149319 15537 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:29:17.149466 15537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "dabd75c6d2d442619e1a65914cec0ebe"
format_stamp: "Formatted at 2026-08-08 14:29:17 on dist-test-slave-w206"
server_key: "1202039672fca61b77fa77a7fcfd93ae"
server_key_iv: "b61d4082b5fbc709511bbbbe946ff43a"
server_key_version: "encryptionkey@0"
I20260808 14:29:17.149583 15537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:17.180266 15537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:17.183893 15537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:17.184134 15537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:17.192360 15537 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:44075
I20260808 14:29:17.192368 15616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:44075 every 8 connection(s)
I20260808 14:29:17.193349 15537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:29:17.196278 15617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:17.201762 15617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe: Bootstrap starting.
I20260808 14:29:17.201967 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 15537
I20260808 14:29:17.202111 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:29:17.202394 7669 external_mini_cluster.cc:1428] Setting key 382829bc58d68c315dd05d8dd6d7b984
I20260808 14:29:17.204406 15617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:17.205147 15617 log.cc:824] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:17.207355 15617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe: No bootstrap required, opened a new log
Aug 08 14:29:17 dist-test-slave-w206 krb5kdc[15524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM
I20260808 14:29:17.211437 15617 raft_consensus.cc:348] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dabd75c6d2d442619e1a65914cec0ebe" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44075 } }
I20260808 14:29:17.211757 15617 raft_consensus.cc:374] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:17.211834 15617 raft_consensus.cc:729] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dabd75c6d2d442619e1a65914cec0ebe, State: Initialized, Role: FOLLOWER
I20260808 14:29:17.212378 15617 consensus_queue.cc:260] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dabd75c6d2d442619e1a65914cec0ebe" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44075 } }
I20260808 14:29:17.212574 15622 raft_consensus.cc:482] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:29:17.212785 15622 raft_consensus.cc:504] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dabd75c6d2d442619e1a65914cec0ebe" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44075 } }
I20260808 14:29:17.212540 15617 raft_consensus.cc:388] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:29:17.213200 15622 leader_election.cc:302] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dabd75c6d2d442619e1a65914cec0ebe; no voters:
I20260808 14:29:17.213280 15617 raft_consensus.cc:482] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:29:17.213351 15617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:17.213678 15622 leader_election.cc:288] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260808 14:29:17.214691 15617 raft_consensus.cc:504] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dabd75c6d2d442619e1a65914cec0ebe" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44075 } }
I20260808 14:29:17.214833 15617 leader_election.cc:302] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dabd75c6d2d442619e1a65914cec0ebe; no voters:
I20260808 14:29:17.214871 15623 raft_consensus.cc:2741] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260808 14:29:17.214939 15617 leader_election.cc:288] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:29:17.215265 15622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:29:17.215561 15622 raft_consensus.cc:686] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe [term 1 LEADER]: Becoming Leader. State: Replica: dabd75c6d2d442619e1a65914cec0ebe, State: Running, Role: LEADER
I20260808 14:29:17.215984 15622 consensus_queue.cc:237] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dabd75c6d2d442619e1a65914cec0ebe" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44075 } }
I20260808 14:29:17.216439 15617 sys_catalog.cc:564] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:29:17.217628 15624 sys_catalog.cc:455] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe [sys.catalog]: SysCatalogTable state changed. Reason: New leader dabd75c6d2d442619e1a65914cec0ebe. Latest consensus state: current_term: 1 leader_uuid: "dabd75c6d2d442619e1a65914cec0ebe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dabd75c6d2d442619e1a65914cec0ebe" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44075 } } }
I20260808 14:29:17.217764 15624 sys_catalog.cc:458] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:17.218149 15623 sys_catalog.cc:455] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dabd75c6d2d442619e1a65914cec0ebe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dabd75c6d2d442619e1a65914cec0ebe" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44075 } } }
I20260808 14:29:17.218288 15623 sys_catalog.cc:458] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:17.218338 15620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:17.203769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35620 (local address 127.7.125.126:44075)
0808 14:29:17.204211 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:17.204227 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:17.204262 (+ 35us) server_negotiation.cc:407] Connection header received
0808 14:29:17.204946 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:17.204964 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:17.205325 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:17.205681 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:17.206488 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.207619 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.208749 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.209219 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.209338 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:17.212160 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:17.212165 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:17.212179 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:17.212208 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:17.214371 (+ 2163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:17.214862 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:17.214869 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:17.214877 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:17.215098 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:17.215477 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:17.215480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:17.215482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:17.215981 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:17.216194 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:17.216841 (+ 647us) server_negotiation.cc:299] Negotiation successful
0808 14:29:17.217154 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":138,"threads_started":1}
W20260808 14:29:17.221730 15639 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:29:17.221798 15639 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:29:17.221875 15633 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:29:17.222708 15633 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:29:17.228291 15633 catalog_manager.cc:1347] Generated new cluster ID: 30650d756c17425b87bde556a7819813
I20260808 14:29:17.228358 15633 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:29:17.244119 15633 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:29:17.245036 15633 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:29:17.248842 15633 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe: Generated new TSK 0
I20260808 14:29:17.249483 15633 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.125.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.125.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.125.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.65.keytab.
Entry for principal oryx/127.7.125.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.65.keytab.
I20260808 14:29:17.282208 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:44075
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.65.keytab
--principal=oryx/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:43391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:17.394511 15645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:17.394810 15645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:17.394896 15645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:17.398875 15645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:17.398970 15645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:17.399056 15645 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 14:29:17.399137 15645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:29:17.404366 15645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:43391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.65.keytab
--principal=oryx/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.125.126:44075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:17.405445 15645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:17.406459 15645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:17.415853 15653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:17.415890 15650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:17.415848 15651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:17.416215 15645 server_base.cc:1034] running on GCE node
I20260808 14:29:17.416630 15645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:17.417294 15645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:17.418506 15645 hybrid_clock.cc:648] HybridClock initialized: now 1786199357418491 us; error 28 us; skew 500 ppm
Aug 08 14:29:17 dist-test-slave-w206 krb5kdc[15524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199357, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:17.421268 15645 init.cc:377] Logged in from keytab as oryx/127.7.125.65@KRBTEST.COM (short username oryx)
I20260808 14:29:17.422163 15645 webserver.cc:460] Webserver started at http://127.7.125.65:39785/ using document root <none> and password file <none>
I20260808 14:29:17.422717 15645 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:17.422804 15645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:17.423092 15645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:17.425103 15645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "e9170b9c02f3427599c7f9ffe62a370b"
format_stamp: "Formatted at 2026-08-08 14:29:17 on dist-test-slave-w206"
server_key: "14125be75e587a05861e1020a491f33d"
server_key_iv: "70b7a0264471602ccf605bf919d96675"
server_key_version: "encryptionkey@0"
I20260808 14:29:17.425666 15645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "e9170b9c02f3427599c7f9ffe62a370b"
format_stamp: "Formatted at 2026-08-08 14:29:17 on dist-test-slave-w206"
server_key: "14125be75e587a05861e1020a491f33d"
server_key_iv: "70b7a0264471602ccf605bf919d96675"
server_key_version: "encryptionkey@0"
I20260808 14:29:17.429376 15645 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 14:29:17.431735 15660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:17.432904 15645 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 14:29:17.433056 15645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "e9170b9c02f3427599c7f9ffe62a370b"
format_stamp: "Formatted at 2026-08-08 14:29:17 on dist-test-slave-w206"
server_key: "14125be75e587a05861e1020a491f33d"
server_key_iv: "70b7a0264471602ccf605bf919d96675"
server_key_version: "encryptionkey@0"
I20260808 14:29:17.433166 15645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:17.448880 15645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:17.451798 15645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:17.452030 15645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:17.453624 15645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:17.453688 15645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:17.453754 15645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:17.453792 15645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 14:29:17 dist-test-slave-w206 krb5kdc[15524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199357, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM
I20260808 14:29:17.465169 15645 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:34197
I20260808 14:29:17.465427 15783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:34197 every 8 connection(s)
I20260808 14:29:17.466161 15645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:29:17.468191 15620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:17.454310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:34017 (local address 127.7.125.126:44075)
0808 14:29:17.454461 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:17.454464 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:17.455535 (+ 1071us) server_negotiation.cc:407] Connection header received
0808 14:29:17.456582 (+ 1047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:17.456586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:17.456647 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:17.456735 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:17.457987 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.458698 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.460122 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.460369 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.460486 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:17.464177 (+ 3691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:17.464181 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:17.464184 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:17.464213 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:17.465966 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:17.466542 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:17.466546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:17.466548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:17.466669 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:17.467076 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:17.467079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:17.467081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:17.467274 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:17.467384 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:17.467885 (+ 501us) server_negotiation.cc:299] Negotiation successful
0808 14:29:17.468017 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 14:29:17.468905 15671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:17.454862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:44075 (local address 127.7.125.65:34017)
0808 14:29:17.455386 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:17.455428 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:17.456370 (+ 942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:17.457030 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:17.457037 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:17.457372 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:17.457825 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:17.457838 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.458851 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.458855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:17.459987 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:17.459997 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.460542 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.460546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:17.460719 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:17.461419 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:17.461449 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:17.463999 (+ 2550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:17.466087 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:17.466093 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:17.466107 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:17.466433 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:17.466780 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:17.466783 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:17.466785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:17.466958 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:17.467391 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:17.467399 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:17.467635 (+ 236us) client_negotiation.cc:769] Sending connection context
0808 14:29:17.467810 (+ 175us) client_negotiation.cc:240] Negotiation successful
0808 14:29:17.468130 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":128,"threads_started":1}
I20260808 14:29:17.469594 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 15645
I20260808 14:29:17.469738 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:29:17.470036 7669 external_mini_cluster.cc:1428] Setting key 3e3871cd7472502fac343a0a8ebbd917
I20260808 14:29:17.470326 15784 heartbeater.cc:347] Connected to a master server at 127.7.125.126:44075
I20260808 14:29:17.470479 15784 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:17.471057 15784 heartbeater.cc:511] Master 127.7.125.126:44075 requested a full tablet report, sending...
I20260808 14:29:17.472528 15570 ts_manager.cc:194] Registered new tserver with Master: e9170b9c02f3427599c7f9ffe62a370b (127.7.125.65:34197)
I20260808 14:29:17.473603 15570 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:34017
WARNING: no policy specified for oryx/127.7.125.66@KRBTEST.COM; defaulting to no policy
I20260808 14:29:17.481215 15620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:17.474204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:55749 (local address 127.7.125.126:44075)
0808 14:29:17.474320 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:17.474324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:17.474449 (+ 125us) server_negotiation.cc:407] Connection header received
0808 14:29:17.474559 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:17.474562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:17.474606 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:17.474687 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:17.475273 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.475778 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.476577 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.476879 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.476992 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:17.478081 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:17.478085 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:17.478087 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:17.478110 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:17.479550 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:17.479981 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:17.479985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:17.479987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:17.480069 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:17.480341 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:17.480346 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:17.480348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:17.480576 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:17.480679 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:17.480951 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 14:29:17.481062 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 14:29:17.481225 15791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:17.473984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:44075 (local address 127.7.125.65:55749)
0808 14:29:17.474355 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:17.474371 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:17.474477 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:17.474730 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:17.474734 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:17.475012 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:17.475161 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:17.475168 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.475912 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.475915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:17.476461 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:17.476468 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.476998 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.477002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:17.477098 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:17.477618 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:17.477630 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:17.477947 (+ 317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:17.479663 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:17.479668 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:17.479671 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:17.479880 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:17.480160 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:17.480164 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:17.480166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:17.480252 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:17.480670 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:17.480673 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:17.480770 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 14:29:17.480893 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 14:29:17.481040 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":110,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.125.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.125.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.66.keytab.
Entry for principal oryx/127.7.125.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.66.keytab.
I20260808 14:29:17.499609 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:44075
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.66.keytab
--principal=oryx/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:43391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:17.613232 15793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:17.613471 15793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:17.613552 15793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:17.618117 15793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:17.618211 15793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:17.618300 15793 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 14:29:17.618373 15793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:29:17.623780 15793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:43391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.66.keytab
--principal=oryx/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.125.126:44075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:17.624825 15793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:17.625836 15793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:17.635528 15799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:17.635571 15793 server_base.cc:1034] running on GCE node
W20260808 14:29:17.635514 15798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:17.635504 15801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:17.636093 15793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:17.636751 15793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:17.637975 15793 hybrid_clock.cc:648] HybridClock initialized: now 1786199357637962 us; error 25 us; skew 500 ppm
Aug 08 14:29:17 dist-test-slave-w206 krb5kdc[15524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199357, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:17.640893 15793 init.cc:377] Logged in from keytab as oryx/127.7.125.66@KRBTEST.COM (short username oryx)
I20260808 14:29:17.641710 15793 webserver.cc:460] Webserver started at http://127.7.125.66:44019/ using document root <none> and password file <none>
I20260808 14:29:17.642257 15793 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:17.642349 15793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:17.642571 15793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:17.644644 15793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "96c5dbfefc7c421d8e7d380fee426780"
format_stamp: "Formatted at 2026-08-08 14:29:17 on dist-test-slave-w206"
server_key: "84deaa52b42c30e0c18fd7bb061b0c4d"
server_key_iv: "15e268fb6e898c04c0226c7a41d799ce"
server_key_version: "encryptionkey@0"
I20260808 14:29:17.645200 15793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "96c5dbfefc7c421d8e7d380fee426780"
format_stamp: "Formatted at 2026-08-08 14:29:17 on dist-test-slave-w206"
server_key: "84deaa52b42c30e0c18fd7bb061b0c4d"
server_key_iv: "15e268fb6e898c04c0226c7a41d799ce"
server_key_version: "encryptionkey@0"
I20260808 14:29:17.648906 15793 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260808 14:29:17.651268 15808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:17.652474 15793 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 14:29:17.652604 15793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "96c5dbfefc7c421d8e7d380fee426780"
format_stamp: "Formatted at 2026-08-08 14:29:17 on dist-test-slave-w206"
server_key: "84deaa52b42c30e0c18fd7bb061b0c4d"
server_key_iv: "15e268fb6e898c04c0226c7a41d799ce"
server_key_version: "encryptionkey@0"
I20260808 14:29:17.652691 15793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:17.667140 15793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:17.670332 15793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:17.670506 15793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:17.672070 15793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:17.672139 15793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:17.672204 15793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:17.672248 15793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 14:29:17 dist-test-slave-w206 krb5kdc[15524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199357, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM
I20260808 14:29:17.683204 15793 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:35199
I20260808 14:29:17.683347 15931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:35199 every 8 connection(s)
I20260808 14:29:17.684351 15793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:29:17.686830 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 15793
I20260808 14:29:17.687016 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:29:17.687371 7669 external_mini_cluster.cc:1428] Setting key aef480789e061acaeba5fd912c312667
I20260808 14:29:17.687393 15620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:17.672849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:58039 (local address 127.7.125.126:44075)
0808 14:29:17.672978 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:17.672981 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:17.674107 (+ 1126us) server_negotiation.cc:407] Connection header received
0808 14:29:17.675242 (+ 1135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:17.675249 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:17.675314 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:17.675379 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:17.677266 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.678187 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.679977 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.680331 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.680469 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:17.683456 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:17.683460 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:17.683464 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:17.683501 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:17.685064 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:17.685602 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:17.685606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:17.685608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:17.685691 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:17.686048 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:17.686051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:17.686053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:17.686278 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:17.686385 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:17.687141 (+ 756us) server_negotiation.cc:299] Negotiation successful
0808 14:29:17.687260 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 14:29:17.688364 15817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:17.673421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:44075 (local address 127.7.125.66:58039)
0808 14:29:17.673933 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:17.673976 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:17.674959 (+ 983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:17.675836 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:17.675845 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:17.676304 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:17.677112 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:17.677130 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.678330 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.678336 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:17.679824 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:17.679835 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.680487 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.680489 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:17.680626 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:17.681286 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:17.681303 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:17.683233 (+ 1930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:17.685219 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:17.685224 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:17.685242 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:17.685489 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:17.685782 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:17.685786 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:17.685788 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:17.685943 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:17.686400 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:17.686409 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:17.686760 (+ 351us) client_negotiation.cc:769] Sending connection context
0808 14:29:17.687024 (+ 264us) client_negotiation.cc:240] Negotiation successful
0808 14:29:17.687383 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":144,"threads_started":1}
I20260808 14:29:17.689572 15932 heartbeater.cc:347] Connected to a master server at 127.7.125.126:44075
I20260808 14:29:17.689723 15932 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:17.690320 15932 heartbeater.cc:511] Master 127.7.125.126:44075 requested a full tablet report, sending...
I20260808 14:29:17.691428 15570 ts_manager.cc:194] Registered new tserver with Master: 96c5dbfefc7c421d8e7d380fee426780 (127.7.125.66:35199)
I20260808 14:29:17.692121 15570 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:58039
WARNING: no policy specified for oryx/127.7.125.67@KRBTEST.COM; defaulting to no policy
I20260808 14:29:17.700232 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:17.691807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:44075 (local address 127.7.125.66:56711)
0808 14:29:17.692105 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:17.692121 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:17.692215 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:17.692762 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:17.692768 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:17.693030 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:17.693187 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:17.693198 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.694208 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.694214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:17.695309 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:17.695324 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.695744 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.695750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:17.695884 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:17.696425 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:17.696440 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:17.696904 (+ 464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:17.698682 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:17.698686 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:17.698688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:17.698961 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:17.699283 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:17.699288 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:17.699290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:17.699384 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:17.699756 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:17.699758 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:17.699832 (+ 74us) client_negotiation.cc:769] Sending connection context
0808 14:29:17.699956 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 14:29:17.700084 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":116,"threads_started":1}
I20260808 14:29:17.700256 15620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:17.692428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:56711 (local address 127.7.125.126:44075)
0808 14:29:17.692548 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:17.692551 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:17.692562 (+ 11us) server_negotiation.cc:407] Connection header received
0808 14:29:17.692607 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:17.692610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:17.692648 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:17.692744 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:17.693330 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.694055 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.695431 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.695631 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.695705 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:17.697034 (+ 1329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:17.697036 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:17.697038 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:17.697060 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:17.698568 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:17.699103 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:17.699106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:17.699107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:17.699185 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:17.699467 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:17.699470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:17.699471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:17.699675 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:17.699753 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:17.700032 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 14:29:17.700147 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.125.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.125.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.67.keytab.
Entry for principal oryx/127.7.125.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.67.keytab.
I20260808 14:29:17.718334 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:44075
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.67.keytab
--principal=oryx/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:43391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:17.832381 15941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:17.832649 15941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:17.832693 15941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:17.836907 15941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:17.836967 15941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:17.837006 15941 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 14:29:17.837065 15941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:29:17.842567 15941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:43391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.67.keytab
--principal=oryx/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.125.126:44075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:17.843720 15941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:17.844738 15941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:17.854441 15949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:17.854452 15947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:17.854449 15946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:17.854672 15941 server_base.cc:1034] running on GCE node
I20260808 14:29:17.855190 15941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:17.855887 15941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:17.857090 15941 hybrid_clock.cc:648] HybridClock initialized: now 1786199357857064 us; error 52 us; skew 500 ppm
Aug 08 14:29:17 dist-test-slave-w206 krb5kdc[15524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199357, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:17.859905 15941 init.cc:377] Logged in from keytab as oryx/127.7.125.67@KRBTEST.COM (short username oryx)
I20260808 14:29:17.860780 15941 webserver.cc:460] Webserver started at http://127.7.125.67:36169/ using document root <none> and password file <none>
I20260808 14:29:17.861354 15941 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:17.861441 15941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:17.861675 15941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:17.863750 15941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "6b06e804fc5f4b59b12f27a41b46e807"
format_stamp: "Formatted at 2026-08-08 14:29:17 on dist-test-slave-w206"
server_key: "8004558d2f00a1c0b900380f4c32e42f"
server_key_iv: "7b7868ca37a185fff7b62cebce3cf739"
server_key_version: "encryptionkey@0"
I20260808 14:29:17.864315 15941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "6b06e804fc5f4b59b12f27a41b46e807"
format_stamp: "Formatted at 2026-08-08 14:29:17 on dist-test-slave-w206"
server_key: "8004558d2f00a1c0b900380f4c32e42f"
server_key_iv: "7b7868ca37a185fff7b62cebce3cf739"
server_key_version: "encryptionkey@0"
I20260808 14:29:17.868119 15941 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 14:29:17.870566 15956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:17.871877 15941 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:29:17.872025 15941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "6b06e804fc5f4b59b12f27a41b46e807"
format_stamp: "Formatted at 2026-08-08 14:29:17 on dist-test-slave-w206"
server_key: "8004558d2f00a1c0b900380f4c32e42f"
server_key_iv: "7b7868ca37a185fff7b62cebce3cf739"
server_key_version: "encryptionkey@0"
I20260808 14:29:17.872126 15941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:17.890467 15941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:17.893939 15941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:17.894143 15941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:17.895699 15941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:17.895753 15941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:17.895812 15941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:17.895854 15941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 14:29:17 dist-test-slave-w206 krb5kdc[15524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199357, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM
I20260808 14:29:17.906563 15941 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:44167
I20260808 14:29:17.906622 16075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:44167 every 8 connection(s)
I20260808 14:29:17.907688 15941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:29:17.909937 15620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:17.896460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:36621 (local address 127.7.125.126:44075)
0808 14:29:17.896634 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:17.896640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:17.897465 (+ 825us) server_negotiation.cc:407] Connection header received
0808 14:29:17.898478 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:17.898485 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:17.898552 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:17.898616 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:17.900459 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.901353 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.903143 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.903532 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.903646 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:17.906428 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:17.906433 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:17.906436 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:17.906469 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:17.908013 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:17.908598 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:17.908601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:17.908602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:17.908686 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:17.909036 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:17.909038 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:17.909039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:17.909212 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:17.909290 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:17.909700 (+ 410us) server_negotiation.cc:299] Negotiation successful
0808 14:29:17.909803 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 14:29:17.910697 15966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:17.896859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:44075 (local address 127.7.125.67:36621)
0808 14:29:17.897295 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:17.897354 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:17.898193 (+ 839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:17.899095 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:17.899105 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:17.899613 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:17.900253 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:17.900272 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.901533 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.901538 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:17.903026 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:17.903036 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.903689 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.903692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:17.903834 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:17.904440 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:17.904465 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:17.906252 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:17.908123 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:17.908129 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:17.908142 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:17.908497 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:17.908774 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:17.908778 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:17.908780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:17.908950 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:17.909297 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:17.909302 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:17.909509 (+ 207us) client_negotiation.cc:769] Sending connection context
0808 14:29:17.909653 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 14:29:17.909892 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":117,"threads_started":1}
I20260808 14:29:17.911895 16076 heartbeater.cc:347] Connected to a master server at 127.7.125.126:44075
I20260808 14:29:17.912051 16076 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:17.912537 16076 heartbeater.cc:511] Master 127.7.125.126:44075 requested a full tablet report, sending...
I20260808 14:29:17.913575 15569 ts_manager.cc:194] Registered new tserver with Master: 6b06e804fc5f4b59b12f27a41b46e807 (127.7.125.67:44167)
I20260808 14:29:17.914320 15569 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:36621
I20260808 14:29:17.915534 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 15941
I20260808 14:29:17.915669 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:29:17.915973 7669 external_mini_cluster.cc:1428] Setting key aa2e7fa7052a8bea932a12256618ce05
I20260808 14:29:17.918092 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:29:17.922067 15620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:17.914119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:50805 (local address 127.7.125.126:44075)
0808 14:29:17.914254 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:17.914258 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:17.914444 (+ 186us) server_negotiation.cc:407] Connection header received
0808 14:29:17.914562 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:17.914567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:17.914624 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:17.914705 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:17.915303 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.916034 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.917169 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.917435 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.917538 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:17.918912 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:17.918915 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:17.918917 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:17.918940 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:17.920456 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:17.920892 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:17.920896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:17.920901 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:17.920984 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:17.921260 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:17.921265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:17.921267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:17.921440 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:17.921545 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:17.921795 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 14:29:17.921906 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 14:29:17.922086 16082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:17.914062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:44075 (local address 127.7.125.67:50805)
0808 14:29:17.914352 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:17.914367 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:17.914465 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:17.914731 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:17.914734 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:17.915061 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:17.915187 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:17.915193 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.916158 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.916162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:17.917053 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:17.917064 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.917571 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.917575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:17.917675 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:17.918240 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:17.918254 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:17.918776 (+ 522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:17.920586 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:17.920589 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:17.920592 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:17.920794 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:17.921081 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:17.921084 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:17.921086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:17.921173 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:17.921537 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:17.921540 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:17.921635 (+ 95us) client_negotiation.cc:769] Sending connection context
0808 14:29:17.921740 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 14:29:17.921884 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":3,"thread_start_us":114,"threads_started":1}
Aug 08 14:29:17 dist-test-slave-w206 krb5kdc[15524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199357, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 14:29:17 dist-test-slave-w206 krb5kdc[15524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:17 dist-test-slave-w206 krb5kdc[15524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 14:29:17.943091 15620 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:17.936910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35634 (local address 127.7.125.126:44075)
0808 14:29:17.937043 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:17.937046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:17.937057 (+ 11us) server_negotiation.cc:407] Connection header received
0808 14:29:17.937095 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:17.937097 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:17.937131 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:17.937214 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:17.937934 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.938508 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.939494 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.939710 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.939816 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:17.942905 (+ 3089us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35634: BlockingRecv error: recv got EOF from 127.0.0.1:35634 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 14:29:17 dist-test-slave-w206 krb5kdc[15524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199357, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM
I20260808 14:29:17.951923 15620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:17.944606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35648 (local address 127.7.125.126:44075)
0808 14:29:17.944716 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:17.944719 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:17.944842 (+ 123us) server_negotiation.cc:407] Connection header received
0808 14:29:17.944937 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:17.944939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:17.944973 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:17.945038 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:17.945562 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.946041 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.946741 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.946938 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.947118 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:17.949286 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:17.949289 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:17.949291 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:17.949333 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:17.950518 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:17.950876 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:17.950878 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:17.950880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:17.950949 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:17.951216 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:17.951218 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:17.951219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:17.951416 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:17.951519 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:17.951722 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 14:29:17.951810 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 14:29:17.954627 15569 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:29:17.957823 15569 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:29:17.971771 16102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:17.966027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:34197 (local address 127.0.0.1:47010)
0808 14:29:17.967038 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:17.967051 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:17.967142 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:17.967901 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:17.967904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:17.967928 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:17.968037 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:17.968044 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.969009 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.969014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:17.970287 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:17.970312 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.971219 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.971223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:17.971338 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:17.971380 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 14:29:17.971499 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 14:29:17.971614 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":925,"thread_start_us":104,"threads_started":1}
I20260808 14:29:17.971773 16103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:17.966593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47010 (local address 127.7.125.65:34197)
0808 14:29:17.967451 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:17.967467 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:17.967489 (+ 22us) server_negotiation.cc:407] Connection header received
0808 14:29:17.967569 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:17.967576 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:17.967757 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:17.967909 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:17.968159 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.968898 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.970414 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.971086 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.971189 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:17.971420 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:17.971527 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 14:29:17.971619 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":74,"threads_started":1}
I20260808 14:29:17.972339 16101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:17.966264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:44167 (local address 127.0.0.1:35268)
0808 14:29:17.967374 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:17.967387 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:17.967484 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:17.967925 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:17.967928 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:17.967956 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:17.968046 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:17.968052 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.969606 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.969609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:17.970757 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:17.970768 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.972005 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.972009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:17.972102 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:17.972125 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 14:29:17.972194 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 14:29:17.972245 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1047,"spinlock_wait_cycles":2816,"thread_start_us":97,"threads_started":1}
I20260808 14:29:17.972476 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:17.966775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35268 (local address 127.7.125.67:44167)
0808 14:29:17.967230 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:17.967241 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:17.967480 (+ 239us) server_negotiation.cc:407] Connection header received
0808 14:29:17.967591 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:17.967596 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:17.967774 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:17.967935 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:17.968159 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.969337 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.970866 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.971675 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.971770 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:17.971938 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:17.972249 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 14:29:17.972346 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":137,"threads_started":1}
I20260808 14:29:17.973412 16099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:17.965837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:35199 (local address 127.0.0.1:48772)
0808 14:29:17.966615 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:17.966668 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:17.966801 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:17.967618 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:17.967624 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:17.967643 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:17.967776 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:17.967783 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.969469 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.969476 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:17.970699 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:17.970710 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.973018 (+ 2308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.973021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:17.973125 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:17.973149 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 14:29:17.973215 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 14:29:17.973288 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":89,"threads_started":1}
I20260808 14:29:17.973929 16100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:17.966029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48772 (local address 127.7.125.66:35199)
0808 14:29:17.966447 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:17.966460 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:17.966952 (+ 492us) server_negotiation.cc:407] Connection header received
0808 14:29:17.967058 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:17.967063 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:17.967427 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:17.967599 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:17.968195 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.969316 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.970842 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:17.971654 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:17.971753 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:17.973712 (+ 1959us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:17.973755 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 14:29:17.973825 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":103,"threads_started":1}
I20260808 14:29:17.974908 16007 tablet_service.cc:1467] Processing CreateTablet for tablet 2b65d37715c44b87a58af926f865dacb (DEFAULT_TABLE table=test-table [id=0679308bc0314b878c46fd22b398add8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:17.974893 15714 tablet_service.cc:1467] Processing CreateTablet for tablet 2b65d37715c44b87a58af926f865dacb (DEFAULT_TABLE table=test-table [id=0679308bc0314b878c46fd22b398add8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:17.976087 16007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b65d37715c44b87a58af926f865dacb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:17.976084 15714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b65d37715c44b87a58af926f865dacb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:17.976404 15863 tablet_service.cc:1467] Processing CreateTablet for tablet 2b65d37715c44b87a58af926f865dacb (DEFAULT_TABLE table=test-table [id=0679308bc0314b878c46fd22b398add8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:17.977408 15863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b65d37715c44b87a58af926f865dacb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:17.983579 16105 tablet_bootstrap.cc:492] T 2b65d37715c44b87a58af926f865dacb P 96c5dbfefc7c421d8e7d380fee426780: Bootstrap starting.
I20260808 14:29:17.984998 16108 tablet_bootstrap.cc:492] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807: Bootstrap starting.
I20260808 14:29:17.984982 16106 tablet_bootstrap.cc:492] T 2b65d37715c44b87a58af926f865dacb P e9170b9c02f3427599c7f9ffe62a370b: Bootstrap starting.
I20260808 14:29:17.985947 16105 tablet_bootstrap.cc:654] T 2b65d37715c44b87a58af926f865dacb P 96c5dbfefc7c421d8e7d380fee426780: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:17.986660 16105 log.cc:824] T 2b65d37715c44b87a58af926f865dacb P 96c5dbfefc7c421d8e7d380fee426780: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:17.987138 16108 tablet_bootstrap.cc:654] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:17.987138 16106 tablet_bootstrap.cc:654] T 2b65d37715c44b87a58af926f865dacb P e9170b9c02f3427599c7f9ffe62a370b: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:17.988047 16108 log.cc:824] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:17.988546 16106 log.cc:824] T 2b65d37715c44b87a58af926f865dacb P e9170b9c02f3427599c7f9ffe62a370b: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:17.988929 16105 tablet_bootstrap.cc:492] T 2b65d37715c44b87a58af926f865dacb P 96c5dbfefc7c421d8e7d380fee426780: No bootstrap required, opened a new log
I20260808 14:29:17.989127 16105 ts_tablet_manager.cc:1397] T 2b65d37715c44b87a58af926f865dacb P 96c5dbfefc7c421d8e7d380fee426780: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 14:29:17.990094 16108 tablet_bootstrap.cc:492] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807: No bootstrap required, opened a new log
I20260808 14:29:17.990285 16108 ts_tablet_manager.cc:1397] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260808 14:29:17.990525 16106 tablet_bootstrap.cc:492] T 2b65d37715c44b87a58af926f865dacb P e9170b9c02f3427599c7f9ffe62a370b: No bootstrap required, opened a new log
I20260808 14:29:17.990700 16106 ts_tablet_manager.cc:1397] T 2b65d37715c44b87a58af926f865dacb P e9170b9c02f3427599c7f9ffe62a370b: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260808 14:29:17.992461 16105 raft_consensus.cc:348] T 2b65d37715c44b87a58af926f865dacb P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:17.992694 16105 raft_consensus.cc:374] T 2b65d37715c44b87a58af926f865dacb P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:17.992748 16105 raft_consensus.cc:729] T 2b65d37715c44b87a58af926f865dacb P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96c5dbfefc7c421d8e7d380fee426780, State: Initialized, Role: FOLLOWER
I20260808 14:29:17.993137 16105 consensus_queue.cc:260] T 2b65d37715c44b87a58af926f865dacb P 96c5dbfefc7c421d8e7d380fee426780 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:17.993818 15932 heartbeater.cc:503] Master 127.7.125.126:44075 was elected leader, sending a full tablet report...
I20260808 14:29:17.994308 16108 raft_consensus.cc:348] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:17.994537 16108 raft_consensus.cc:374] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:17.994604 16108 raft_consensus.cc:729] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b06e804fc5f4b59b12f27a41b46e807, State: Initialized, Role: FOLLOWER
I20260808 14:29:17.994560 16106 raft_consensus.cc:348] T 2b65d37715c44b87a58af926f865dacb P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:17.994786 16106 raft_consensus.cc:374] T 2b65d37715c44b87a58af926f865dacb P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:17.994844 16106 raft_consensus.cc:729] T 2b65d37715c44b87a58af926f865dacb P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9170b9c02f3427599c7f9ffe62a370b, State: Initialized, Role: FOLLOWER
I20260808 14:29:17.995090 16105 ts_tablet_manager.cc:1428] T 2b65d37715c44b87a58af926f865dacb P 96c5dbfefc7c421d8e7d380fee426780: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 14:29:17.995081 16108 consensus_queue.cc:260] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:17.995257 16106 consensus_queue.cc:260] T 2b65d37715c44b87a58af926f865dacb P e9170b9c02f3427599c7f9ffe62a370b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:17.995718 16076 heartbeater.cc:503] Master 127.7.125.126:44075 was elected leader, sending a full tablet report...
I20260808 14:29:17.996099 16106 ts_tablet_manager.cc:1428] T 2b65d37715c44b87a58af926f865dacb P e9170b9c02f3427599c7f9ffe62a370b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 14:29:17.996483 15784 heartbeater.cc:503] Master 127.7.125.126:44075 was elected leader, sending a full tablet report...
I20260808 14:29:17.996984 16108 ts_tablet_manager.cc:1428] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 14:29:18.145591 16113 raft_consensus.cc:482] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:29:18.145875 16113 raft_consensus.cc:504] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.147289 16113 leader_election.cc:288] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96c5dbfefc7c421d8e7d380fee426780 (127.7.125.66:35199), e9170b9c02f3427599c7f9ffe62a370b (127.7.125.65:34197)
I20260808 14:29:18.151175 16115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:18.147586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:34197 (local address 127.7.125.67:41965)
0808 14:29:18.147988 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:18.147999 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:18.148128 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:18.148378 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:18.148381 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:18.148395 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:18.148525 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:18.148530 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.149245 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.149250 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:18.150039 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:18.150047 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.150687 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.150691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:18.150807 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:18.150831 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 14:29:18.150917 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 14:29:18.151008 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":137,"threads_started":1}
I20260808 14:29:18.151213 16103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:18.147686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:41965 (local address 127.7.125.65:34197)
0808 14:29:18.147813 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:18.147818 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:18.148082 (+ 264us) server_negotiation.cc:407] Connection header received
0808 14:29:18.148238 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:18.148241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:18.148284 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:18.148367 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:18.148633 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.149135 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.150161 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.150567 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.150711 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:18.150771 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:18.150951 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 14:29:18.151065 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:29:18.151866 15966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:18.147526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:35199 (local address 127.7.125.67:34603)
0808 14:29:18.147669 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:18.147688 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:18.147798 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:18.148133 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:18.148136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:18.148155 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:18.148302 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:18.148308 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.149096 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.149108 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:18.150410 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:18.150421 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.151418 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.151422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:18.151522 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:18.151548 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 14:29:18.151630 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 14:29:18.151701 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 14:29:18.151850 15735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b65d37715c44b87a58af926f865dacb" candidate_uuid: "6b06e804fc5f4b59b12f27a41b46e807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9170b9c02f3427599c7f9ffe62a370b" is_pre_election: true
I20260808 14:29:18.152179 15735 raft_consensus.cc:2454] T 2b65d37715c44b87a58af926f865dacb P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b06e804fc5f4b59b12f27a41b46e807 in term 0.
I20260808 14:29:18.152253 16100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:18.147759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:34603 (local address 127.7.125.66:35199)
0808 14:29:18.147895 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:18.147902 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:18.147920 (+ 18us) server_negotiation.cc:407] Connection header received
0808 14:29:18.147983 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:18.147985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:18.148025 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:18.148119 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:18.148432 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.148968 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.150556 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.151182 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.151317 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:18.151382 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:18.152023 (+ 641us) server_negotiation.cc:299] Negotiation successful
0808 14:29:18.152115 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:29:18.152725 15961 leader_election.cc:302] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b06e804fc5f4b59b12f27a41b46e807, e9170b9c02f3427599c7f9ffe62a370b; no voters:
I20260808 14:29:18.152768 15884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b65d37715c44b87a58af926f865dacb" candidate_uuid: "6b06e804fc5f4b59b12f27a41b46e807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96c5dbfefc7c421d8e7d380fee426780" is_pre_election: true
I20260808 14:29:18.153018 16113 raft_consensus.cc:2781] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:29:18.153072 15884 raft_consensus.cc:2454] T 2b65d37715c44b87a58af926f865dacb P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b06e804fc5f4b59b12f27a41b46e807 in term 0.
I20260808 14:29:18.153142 16113 raft_consensus.cc:482] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:29:18.153187 16113 raft_consensus.cc:3037] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:18.154318 16113 raft_consensus.cc:504] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.154656 16113 leader_election.cc:288] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807 [CANDIDATE]: Term 1 election: Requested vote from peers 96c5dbfefc7c421d8e7d380fee426780 (127.7.125.66:35199), e9170b9c02f3427599c7f9ffe62a370b (127.7.125.65:34197)
I20260808 14:29:18.155035 15884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b65d37715c44b87a58af926f865dacb" candidate_uuid: "6b06e804fc5f4b59b12f27a41b46e807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96c5dbfefc7c421d8e7d380fee426780"
I20260808 14:29:18.155042 15735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b65d37715c44b87a58af926f865dacb" candidate_uuid: "6b06e804fc5f4b59b12f27a41b46e807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9170b9c02f3427599c7f9ffe62a370b"
I20260808 14:29:18.155172 15735 raft_consensus.cc:3037] T 2b65d37715c44b87a58af926f865dacb P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:18.155241 15884 raft_consensus.cc:3037] T 2b65d37715c44b87a58af926f865dacb P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:18.156623 15884 raft_consensus.cc:2454] T 2b65d37715c44b87a58af926f865dacb P 96c5dbfefc7c421d8e7d380fee426780 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b06e804fc5f4b59b12f27a41b46e807 in term 1.
I20260808 14:29:18.156623 15735 raft_consensus.cc:2454] T 2b65d37715c44b87a58af926f865dacb P e9170b9c02f3427599c7f9ffe62a370b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b06e804fc5f4b59b12f27a41b46e807 in term 1.
I20260808 14:29:18.156956 15958 leader_election.cc:302] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b06e804fc5f4b59b12f27a41b46e807, 96c5dbfefc7c421d8e7d380fee426780; no voters:
I20260808 14:29:18.157184 16113 raft_consensus.cc:2781] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:29:18.157454 16113 raft_consensus.cc:686] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807 [term 1 LEADER]: Becoming Leader. State: Replica: 6b06e804fc5f4b59b12f27a41b46e807, State: Running, Role: LEADER
I20260808 14:29:18.157781 16113 consensus_queue.cc:237] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
W20260808 14:29:18.159116 16077 tablet.cc:2369] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:29:18.161461 15569 catalog_manager.cc:5495] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b06e804fc5f4b59b12f27a41b46e807 (127.7.125.67). New cstate: current_term: 1 leader_uuid: "6b06e804fc5f4b59b12f27a41b46e807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } health_report { overall_health: HEALTHY } } }
W20260808 14:29:18.185652 15933 tablet.cc:2369] T 2b65d37715c44b87a58af926f865dacb P 96c5dbfefc7c421d8e7d380fee426780: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:29:18.205772 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:18.202219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35276 (local address 127.7.125.67:44167)
0808 14:29:18.202361 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:18.202367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:18.202382 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:29:18.202494 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:18.202496 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:18.202550 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:18.202652 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:29:18.202887 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.203499 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.204361 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.204597 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.204698 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:18.204981 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:29:18.205382 (+ 401us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:29:18.205512 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:18.205576 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 14:29:18.205632 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 14:29:18.211632 15735 raft_consensus.cc:1261] T 2b65d37715c44b87a58af926f865dacb P e9170b9c02f3427599c7f9ffe62a370b [term 1 FOLLOWER]: Refusing update from remote peer 6b06e804fc5f4b59b12f27a41b46e807: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:29:18.211632 15884 raft_consensus.cc:1261] T 2b65d37715c44b87a58af926f865dacb P 96c5dbfefc7c421d8e7d380fee426780 [term 1 FOLLOWER]: Refusing update from remote peer 6b06e804fc5f4b59b12f27a41b46e807: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:29:18.212431 16116 consensus_queue.cc:1035] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:29:18.212639 16113 consensus_queue.cc:1035] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 14:29:18.217669 15785 tablet.cc:2369] T 2b65d37715c44b87a58af926f865dacb P e9170b9c02f3427599c7f9ffe62a370b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:29:18.219362 16119 mvcc.cc:205] Tried to move back new op lower bound from 7316272571226800128 to 7316272571017003008. Current Snapshot: MvccSnapshot[applied={T|T < 7316272571226800128}]
I20260808 14:29:18.220525 16120 mvcc.cc:205] Tried to move back new op lower bound from 7316272571226800128 to 7316272571017003008. Current Snapshot: MvccSnapshot[applied={T|T < 7316272571226800128}]
I20260808 14:29:18.231017 15570 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:29:18.231237 15570 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:29:18.234385 15570 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe: Sending DeleteTablet for 3 replicas of tablet 2b65d37715c44b87a58af926f865dacb
I20260808 14:29:18.236095 16007 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b65d37715c44b87a58af926f865dacb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:18 UTC) from {username='oryx', principal='oryx/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:35268
I20260808 14:29:18.235625 15714 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b65d37715c44b87a58af926f865dacb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:18 UTC) from {username='oryx', principal='oryx/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:47010
I20260808 14:29:18.237121 16128 tablet_replica.cc:330] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807: stopping tablet replica
I20260808 14:29:18.237437 16129 tablet_replica.cc:330] T 2b65d37715c44b87a58af926f865dacb P e9170b9c02f3427599c7f9ffe62a370b: stopping tablet replica
I20260808 14:29:18.237507 16128 raft_consensus.cc:2229] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:29:18.237890 16129 raft_consensus.cc:2229] T 2b65d37715c44b87a58af926f865dacb P e9170b9c02f3427599c7f9ffe62a370b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:29:18.237950 16128 raft_consensus.cc:2258] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:18.238430 16129 raft_consensus.cc:2258] T 2b65d37715c44b87a58af926f865dacb P e9170b9c02f3427599c7f9ffe62a370b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:18.238560 15570 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:29:18.239037 15570 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:29:18.235447 15863 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b65d37715c44b87a58af926f865dacb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:18 UTC) from {username='oryx', principal='oryx/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:48772
I20260808 14:29:18.239449 16130 tablet_replica.cc:330] T 2b65d37715c44b87a58af926f865dacb P 96c5dbfefc7c421d8e7d380fee426780: stopping tablet replica
I20260808 14:29:18.240554 16129 ts_tablet_manager.cc:1905] T 2b65d37715c44b87a58af926f865dacb P e9170b9c02f3427599c7f9ffe62a370b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:29:18.240643 16128 ts_tablet_manager.cc:1905] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:29:18.241281 16130 raft_consensus.cc:2229] T 2b65d37715c44b87a58af926f865dacb P 96c5dbfefc7c421d8e7d380fee426780 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:29:18.241595 16130 raft_consensus.cc:2258] T 2b65d37715c44b87a58af926f865dacb P 96c5dbfefc7c421d8e7d380fee426780 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:18.244341 15714 tablet_service.cc:1467] Processing CreateTablet for tablet 8f941147637a427c8c6119bfa15018d0 (DEFAULT_TABLE table=test-table [id=cceab87ff57449528777f9133218bdc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:18.244655 15714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f941147637a427c8c6119bfa15018d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:18.244756 16007 tablet_service.cc:1467] Processing CreateTablet for tablet 8f941147637a427c8c6119bfa15018d0 (DEFAULT_TABLE table=test-table [id=cceab87ff57449528777f9133218bdc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:18.245050 16007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f941147637a427c8c6119bfa15018d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:18.245563 15863 tablet_service.cc:1467] Processing CreateTablet for tablet 8f941147637a427c8c6119bfa15018d0 (DEFAULT_TABLE table=test-table [id=cceab87ff57449528777f9133218bdc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:18.245810 15863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f941147637a427c8c6119bfa15018d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:18.246178 16128 ts_tablet_manager.cc:1918] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:29:18.246266 16128 log.cc:1196] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/2b65d37715c44b87a58af926f865dacb
I20260808 14:29:18.246577 16128 ts_tablet_manager.cc:1939] T 2b65d37715c44b87a58af926f865dacb P 6b06e804fc5f4b59b12f27a41b46e807: Deleting consensus metadata
I20260808 14:29:18.247422 16108 tablet_bootstrap.cc:492] T 8f941147637a427c8c6119bfa15018d0 P 6b06e804fc5f4b59b12f27a41b46e807: Bootstrap starting.
I20260808 14:29:18.247876 16130 ts_tablet_manager.cc:1905] T 2b65d37715c44b87a58af926f865dacb P 96c5dbfefc7c421d8e7d380fee426780: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:29:18.248402 16108 tablet_bootstrap.cc:654] T 8f941147637a427c8c6119bfa15018d0 P 6b06e804fc5f4b59b12f27a41b46e807: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:18.248973 16105 tablet_bootstrap.cc:492] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780: Bootstrap starting.
I20260808 14:29:18.249223 15553 catalog_manager.cc:4841] TS 6b06e804fc5f4b59b12f27a41b46e807 (127.7.125.67:44167): tablet 2b65d37715c44b87a58af926f865dacb (table test-table [id=0679308bc0314b878c46fd22b398add8]) successfully deleted
I20260808 14:29:18.249887 16105 tablet_bootstrap.cc:654] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:18.250408 16130 ts_tablet_manager.cc:1918] T 2b65d37715c44b87a58af926f865dacb P 96c5dbfefc7c421d8e7d380fee426780: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:29:18.250517 16130 log.cc:1196] T 2b65d37715c44b87a58af926f865dacb P 96c5dbfefc7c421d8e7d380fee426780: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/2b65d37715c44b87a58af926f865dacb
I20260808 14:29:18.250841 16130 ts_tablet_manager.cc:1939] T 2b65d37715c44b87a58af926f865dacb P 96c5dbfefc7c421d8e7d380fee426780: Deleting consensus metadata
I20260808 14:29:18.251366 16106 tablet_bootstrap.cc:492] T 8f941147637a427c8c6119bfa15018d0 P e9170b9c02f3427599c7f9ffe62a370b: Bootstrap starting.
I20260808 14:29:18.251639 16105 tablet_bootstrap.cc:492] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780: No bootstrap required, opened a new log
I20260808 14:29:18.251713 16105 ts_tablet_manager.cc:1397] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:29:18.252089 15554 catalog_manager.cc:4841] TS 96c5dbfefc7c421d8e7d380fee426780 (127.7.125.66:35199): tablet 2b65d37715c44b87a58af926f865dacb (table test-table [id=0679308bc0314b878c46fd22b398add8]) successfully deleted
I20260808 14:29:18.252170 16105 raft_consensus.cc:348] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.252301 16105 raft_consensus.cc:374] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:18.252341 16106 tablet_bootstrap.cc:654] T 8f941147637a427c8c6119bfa15018d0 P e9170b9c02f3427599c7f9ffe62a370b: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:18.252341 16105 raft_consensus.cc:729] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96c5dbfefc7c421d8e7d380fee426780, State: Initialized, Role: FOLLOWER
I20260808 14:29:18.253150 16108 tablet_bootstrap.cc:492] T 8f941147637a427c8c6119bfa15018d0 P 6b06e804fc5f4b59b12f27a41b46e807: No bootstrap required, opened a new log
I20260808 14:29:18.253242 16108 ts_tablet_manager.cc:1397] T 8f941147637a427c8c6119bfa15018d0 P 6b06e804fc5f4b59b12f27a41b46e807: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260808 14:29:18.253492 16129 ts_tablet_manager.cc:1918] T 2b65d37715c44b87a58af926f865dacb P e9170b9c02f3427599c7f9ffe62a370b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:29:18.253580 16129 log.cc:1196] T 2b65d37715c44b87a58af926f865dacb P e9170b9c02f3427599c7f9ffe62a370b: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-0/wal/wals/2b65d37715c44b87a58af926f865dacb
I20260808 14:29:18.253744 16106 tablet_bootstrap.cc:492] T 8f941147637a427c8c6119bfa15018d0 P e9170b9c02f3427599c7f9ffe62a370b: No bootstrap required, opened a new log
I20260808 14:29:18.253818 16106 ts_tablet_manager.cc:1397] T 8f941147637a427c8c6119bfa15018d0 P e9170b9c02f3427599c7f9ffe62a370b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 14:29:18.253726 16108 raft_consensus.cc:348] T 8f941147637a427c8c6119bfa15018d0 P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.253865 16108 raft_consensus.cc:374] T 8f941147637a427c8c6119bfa15018d0 P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:18.253917 16108 raft_consensus.cc:729] T 8f941147637a427c8c6119bfa15018d0 P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b06e804fc5f4b59b12f27a41b46e807, State: Initialized, Role: FOLLOWER
I20260808 14:29:18.254047 16108 consensus_queue.cc:260] T 8f941147637a427c8c6119bfa15018d0 P 6b06e804fc5f4b59b12f27a41b46e807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.254307 16108 ts_tablet_manager.cc:1428] T 8f941147637a427c8c6119bfa15018d0 P 6b06e804fc5f4b59b12f27a41b46e807: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:29:18.254323 16129 ts_tablet_manager.cc:1939] T 2b65d37715c44b87a58af926f865dacb P e9170b9c02f3427599c7f9ffe62a370b: Deleting consensus metadata
I20260808 14:29:18.254280 16106 raft_consensus.cc:348] T 8f941147637a427c8c6119bfa15018d0 P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.254383 16106 raft_consensus.cc:374] T 8f941147637a427c8c6119bfa15018d0 P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:18.254439 16106 raft_consensus.cc:729] T 8f941147637a427c8c6119bfa15018d0 P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9170b9c02f3427599c7f9ffe62a370b, State: Initialized, Role: FOLLOWER
I20260808 14:29:18.254555 16106 consensus_queue.cc:260] T 8f941147637a427c8c6119bfa15018d0 P e9170b9c02f3427599c7f9ffe62a370b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.254792 16106 ts_tablet_manager.cc:1428] T 8f941147637a427c8c6119bfa15018d0 P e9170b9c02f3427599c7f9ffe62a370b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:29:18.255612 16105 consensus_queue.cc:260] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.255784 15556 catalog_manager.cc:4841] TS e9170b9c02f3427599c7f9ffe62a370b (127.7.125.65:34197): tablet 2b65d37715c44b87a58af926f865dacb (table test-table [id=0679308bc0314b878c46fd22b398add8]) successfully deleted
I20260808 14:29:18.255942 16105 ts_tablet_manager.cc:1428] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260808 14:29:18.365761 16112 raft_consensus.cc:482] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:29:18.365932 16112 raft_consensus.cc:504] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.367166 16112 leader_election.cc:288] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9170b9c02f3427599c7f9ffe62a370b (127.7.125.65:34197), 6b06e804fc5f4b59b12f27a41b46e807 (127.7.125.67:44167)
I20260808 14:29:18.371994 16134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:18.367364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:34197 (local address 127.7.125.66:40379)
0808 14:29:18.367753 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:18.367769 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:18.367938 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:18.368226 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:18.368229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:18.368247 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:18.368365 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:18.368370 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.369318 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.369322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:18.370667 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:18.370678 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.371534 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.371537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:18.371643 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:18.371669 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 14:29:18.371752 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 14:29:18.371825 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":21,"thread_start_us":122,"threads_started":1}
I20260808 14:29:18.372008 16103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:18.367572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:40379 (local address 127.7.125.65:34197)
0808 14:29:18.367709 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:18.367713 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:18.367851 (+ 138us) server_negotiation.cc:407] Connection header received
0808 14:29:18.368057 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:18.368060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:18.368113 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:18.368202 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:18.368491 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.369184 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.370794 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.371395 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.371524 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:18.371579 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:18.371782 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 14:29:18.371881 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:29:18.372107 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:18.367822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:59819 (local address 127.7.125.67:44167)
0808 14:29:18.367960 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:18.367964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:18.368095 (+ 131us) server_negotiation.cc:407] Connection header received
0808 14:29:18.368274 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:18.368278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:18.368345 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:18.368399 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:18.368803 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.369436 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.370694 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.371387 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.371524 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:18.371579 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:18.371923 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 14:29:18.371996 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 14:29:18.372447 16135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:18.367478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:44167 (local address 127.7.125.66:59819)
0808 14:29:18.367994 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:18.368009 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:18.368127 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:18.368545 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:18.368548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:18.368565 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:18.368684 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:18.368690 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.369544 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.369549 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:18.370573 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:18.370583 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.371660 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.371663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:18.371765 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:18.371790 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 14:29:18.372281 (+ 491us) client_negotiation.cc:240] Negotiation successful
0808 14:29:18.372342 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":19,"thread_start_us":122,"threads_started":1}
I20260808 14:29:18.373064 15735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f941147637a427c8c6119bfa15018d0" candidate_uuid: "96c5dbfefc7c421d8e7d380fee426780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9170b9c02f3427599c7f9ffe62a370b" is_pre_election: true
I20260808 14:29:18.373224 15735 raft_consensus.cc:2454] T 8f941147637a427c8c6119bfa15018d0 P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96c5dbfefc7c421d8e7d380fee426780 in term 0.
I20260808 14:29:18.373525 16027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f941147637a427c8c6119bfa15018d0" candidate_uuid: "96c5dbfefc7c421d8e7d380fee426780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b06e804fc5f4b59b12f27a41b46e807" is_pre_election: true
I20260808 14:29:18.373594 15812 leader_election.cc:302] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96c5dbfefc7c421d8e7d380fee426780, e9170b9c02f3427599c7f9ffe62a370b; no voters:
I20260808 14:29:18.373730 16027 raft_consensus.cc:2454] T 8f941147637a427c8c6119bfa15018d0 P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96c5dbfefc7c421d8e7d380fee426780 in term 0.
I20260808 14:29:18.373909 16112 raft_consensus.cc:2781] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:29:18.373980 16112 raft_consensus.cc:482] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:29:18.374054 16112 raft_consensus.cc:3037] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:18.375214 16112 raft_consensus.cc:504] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.375669 16112 leader_election.cc:288] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780 [CANDIDATE]: Term 1 election: Requested vote from peers e9170b9c02f3427599c7f9ffe62a370b (127.7.125.65:34197), 6b06e804fc5f4b59b12f27a41b46e807 (127.7.125.67:44167)
I20260808 14:29:18.376048 16027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f941147637a427c8c6119bfa15018d0" candidate_uuid: "96c5dbfefc7c421d8e7d380fee426780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b06e804fc5f4b59b12f27a41b46e807"
I20260808 14:29:18.376106 15735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f941147637a427c8c6119bfa15018d0" candidate_uuid: "96c5dbfefc7c421d8e7d380fee426780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9170b9c02f3427599c7f9ffe62a370b"
I20260808 14:29:18.376199 16027 raft_consensus.cc:3037] T 8f941147637a427c8c6119bfa15018d0 P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:18.376245 15735 raft_consensus.cc:3037] T 8f941147637a427c8c6119bfa15018d0 P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:18.377278 16027 raft_consensus.cc:2454] T 8f941147637a427c8c6119bfa15018d0 P 6b06e804fc5f4b59b12f27a41b46e807 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96c5dbfefc7c421d8e7d380fee426780 in term 1.
I20260808 14:29:18.377394 15735 raft_consensus.cc:2454] T 8f941147637a427c8c6119bfa15018d0 P e9170b9c02f3427599c7f9ffe62a370b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96c5dbfefc7c421d8e7d380fee426780 in term 1.
I20260808 14:29:18.377580 15809 leader_election.cc:302] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b06e804fc5f4b59b12f27a41b46e807, 96c5dbfefc7c421d8e7d380fee426780; no voters:
I20260808 14:29:18.377797 16112 raft_consensus.cc:2781] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:29:18.378031 16112 raft_consensus.cc:686] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780 [term 1 LEADER]: Becoming Leader. State: Replica: 96c5dbfefc7c421d8e7d380fee426780, State: Running, Role: LEADER
I20260808 14:29:18.378273 16112 consensus_queue.cc:237] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.380745 15569 catalog_manager.cc:5495] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96c5dbfefc7c421d8e7d380fee426780 (127.7.125.66). New cstate: current_term: 1 leader_uuid: "96c5dbfefc7c421d8e7d380fee426780" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } health_report { overall_health: UNKNOWN } } }
Aug 08 14:29:18 dist-test-slave-w206 krb5kdc[15524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199357, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.126@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM
I20260808 14:29:18.433014 15620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:18.424749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35654 (local address 127.7.125.126:44075)
0808 14:29:18.424913 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:18.424918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:18.425163 (+ 245us) server_negotiation.cc:407] Connection header received
0808 14:29:18.425266 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:18.425269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:18.425310 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:18.425377 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:18.426063 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.426584 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.427399 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.427600 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.427675 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:18.429949 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:18.429952 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:18.429954 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:18.429974 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:18.431353 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:18.431803 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:18.431806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:18.431807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:18.431886 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:18.432115 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:18.432117 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:18.432118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:18.432277 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:18.432357 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:18.432615 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 14:29:18.432799 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":72}
I20260808 14:29:18.433035 16143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:18.424706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:44075 (local address 127.0.0.1:35654)
0808 14:29:18.425074 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:18.425087 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:18.425188 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:18.425411 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:18.425413 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:18.425849 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:18.425960 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:18.425966 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.426672 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.426674 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:18.427298 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:18.427307 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.427693 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.427695 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:18.427767 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:18.428367 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:18.428378 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:18.429827 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:18.431451 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:18.431460 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:18.431465 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:18.431717 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:18.431961 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:18.431963 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:18.431965 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:18.432043 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:18.432366 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:18.432371 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:18.432469 (+ 98us) client_negotiation.cc:769] Sending connection context
0808 14:29:18.432561 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 14:29:18.432748 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":147,"threads_started":1}
I20260808 14:29:18.437237 15569 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:35654:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 14:29:18.438071 15569 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:29:18.443334 15863 tablet_service.cc:1467] Processing CreateTablet for tablet 038e03baa28e490b998cd9af64dcd9f8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bf7d9a445d2c40978f1e180249cced08]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 14:29:18.443338 16007 tablet_service.cc:1467] Processing CreateTablet for tablet 038e03baa28e490b998cd9af64dcd9f8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bf7d9a445d2c40978f1e180249cced08]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 14:29:18.443807 15863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 038e03baa28e490b998cd9af64dcd9f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:18.443961 16007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 038e03baa28e490b998cd9af64dcd9f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:18.446456 16105 tablet_bootstrap.cc:492] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780: Bootstrap starting.
I20260808 14:29:18.446519 16108 tablet_bootstrap.cc:492] T 038e03baa28e490b998cd9af64dcd9f8 P 6b06e804fc5f4b59b12f27a41b46e807: Bootstrap starting.
I20260808 14:29:18.446800 15714 tablet_service.cc:1467] Processing CreateTablet for tablet 038e03baa28e490b998cd9af64dcd9f8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bf7d9a445d2c40978f1e180249cced08]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 14:29:18.447175 15714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 038e03baa28e490b998cd9af64dcd9f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:18.447443 16108 tablet_bootstrap.cc:654] T 038e03baa28e490b998cd9af64dcd9f8 P 6b06e804fc5f4b59b12f27a41b46e807: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:18.447501 16105 tablet_bootstrap.cc:654] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:18.448843 16108 tablet_bootstrap.cc:492] T 038e03baa28e490b998cd9af64dcd9f8 P 6b06e804fc5f4b59b12f27a41b46e807: No bootstrap required, opened a new log
I20260808 14:29:18.448967 16108 ts_tablet_manager.cc:1397] T 038e03baa28e490b998cd9af64dcd9f8 P 6b06e804fc5f4b59b12f27a41b46e807: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 14:29:18.449091 16105 tablet_bootstrap.cc:492] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780: No bootstrap required, opened a new log
I20260808 14:29:18.449184 16105 ts_tablet_manager.cc:1397] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:29:18.449455 16108 raft_consensus.cc:348] T 038e03baa28e490b998cd9af64dcd9f8 P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.449690 16108 raft_consensus.cc:374] T 038e03baa28e490b998cd9af64dcd9f8 P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:18.449774 16105 raft_consensus.cc:348] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.449878 16105 raft_consensus.cc:374] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:18.449884 16108 raft_consensus.cc:729] T 038e03baa28e490b998cd9af64dcd9f8 P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b06e804fc5f4b59b12f27a41b46e807, State: Initialized, Role: FOLLOWER
I20260808 14:29:18.449924 16105 raft_consensus.cc:729] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96c5dbfefc7c421d8e7d380fee426780, State: Initialized, Role: FOLLOWER
I20260808 14:29:18.450071 16105 consensus_queue.cc:260] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.450054 16108 consensus_queue.cc:260] T 038e03baa28e490b998cd9af64dcd9f8 P 6b06e804fc5f4b59b12f27a41b46e807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.450309 16112 tablet_replica.cc:439] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } } }
I20260808 14:29:18.450445 16112 tablet_replica.cc:442] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:29:18.450461 16108 ts_tablet_manager.cc:1428] T 038e03baa28e490b998cd9af64dcd9f8 P 6b06e804fc5f4b59b12f27a41b46e807: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260808 14:29:18.450696 16116 tablet_replica.cc:439] T 038e03baa28e490b998cd9af64dcd9f8 P 6b06e804fc5f4b59b12f27a41b46e807: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } } }
I20260808 14:29:18.450820 16116 tablet_replica.cc:442] T 038e03baa28e490b998cd9af64dcd9f8 P 6b06e804fc5f4b59b12f27a41b46e807: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:29:18.451252 16105 ts_tablet_manager.cc:1428] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 14:29:18.451373 16106 tablet_bootstrap.cc:492] T 038e03baa28e490b998cd9af64dcd9f8 P e9170b9c02f3427599c7f9ffe62a370b: Bootstrap starting.
I20260808 14:29:18.452554 16106 tablet_bootstrap.cc:654] T 038e03baa28e490b998cd9af64dcd9f8 P e9170b9c02f3427599c7f9ffe62a370b: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:18.454568 16106 tablet_bootstrap.cc:492] T 038e03baa28e490b998cd9af64dcd9f8 P e9170b9c02f3427599c7f9ffe62a370b: No bootstrap required, opened a new log
I20260808 14:29:18.454694 16106 ts_tablet_manager.cc:1397] T 038e03baa28e490b998cd9af64dcd9f8 P e9170b9c02f3427599c7f9ffe62a370b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:29:18.455303 16106 raft_consensus.cc:348] T 038e03baa28e490b998cd9af64dcd9f8 P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.455452 16106 raft_consensus.cc:374] T 038e03baa28e490b998cd9af64dcd9f8 P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:18.455518 16106 raft_consensus.cc:729] T 038e03baa28e490b998cd9af64dcd9f8 P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9170b9c02f3427599c7f9ffe62a370b, State: Initialized, Role: FOLLOWER
I20260808 14:29:18.455663 16106 consensus_queue.cc:260] T 038e03baa28e490b998cd9af64dcd9f8 P e9170b9c02f3427599c7f9ffe62a370b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.456086 16106 ts_tablet_manager.cc:1428] T 038e03baa28e490b998cd9af64dcd9f8 P e9170b9c02f3427599c7f9ffe62a370b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 14:29:18.456044 16114 tablet_replica.cc:439] T 038e03baa28e490b998cd9af64dcd9f8 P e9170b9c02f3427599c7f9ffe62a370b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } } }
I20260808 14:29:18.456197 16114 tablet_replica.cc:442] T 038e03baa28e490b998cd9af64dcd9f8 P e9170b9c02f3427599c7f9ffe62a370b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:29:18.661958 16112 raft_consensus.cc:482] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:29:18.662159 16112 raft_consensus.cc:504] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.662698 16112 leader_election.cc:288] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9170b9c02f3427599c7f9ffe62a370b (127.7.125.65:34197), 6b06e804fc5f4b59b12f27a41b46e807 (127.7.125.67:44167)
I20260808 14:29:18.663194 15735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "038e03baa28e490b998cd9af64dcd9f8" candidate_uuid: "96c5dbfefc7c421d8e7d380fee426780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9170b9c02f3427599c7f9ffe62a370b" is_pre_election: true
I20260808 14:29:18.663268 16027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "038e03baa28e490b998cd9af64dcd9f8" candidate_uuid: "96c5dbfefc7c421d8e7d380fee426780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b06e804fc5f4b59b12f27a41b46e807" is_pre_election: true
I20260808 14:29:18.663336 15735 raft_consensus.cc:2454] T 038e03baa28e490b998cd9af64dcd9f8 P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96c5dbfefc7c421d8e7d380fee426780 in term 0.
I20260808 14:29:18.663473 16027 raft_consensus.cc:2454] T 038e03baa28e490b998cd9af64dcd9f8 P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96c5dbfefc7c421d8e7d380fee426780 in term 0.
I20260808 14:29:18.663744 15812 leader_election.cc:302] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96c5dbfefc7c421d8e7d380fee426780, e9170b9c02f3427599c7f9ffe62a370b; no voters:
I20260808 14:29:18.663919 16112 raft_consensus.cc:2781] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:29:18.663981 16112 raft_consensus.cc:482] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:29:18.664023 16112 raft_consensus.cc:3037] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:18.664961 16112 raft_consensus.cc:504] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.665280 16112 leader_election.cc:288] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780 [CANDIDATE]: Term 1 election: Requested vote from peers e9170b9c02f3427599c7f9ffe62a370b (127.7.125.65:34197), 6b06e804fc5f4b59b12f27a41b46e807 (127.7.125.67:44167)
I20260808 14:29:18.665685 15735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "038e03baa28e490b998cd9af64dcd9f8" candidate_uuid: "96c5dbfefc7c421d8e7d380fee426780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9170b9c02f3427599c7f9ffe62a370b"
I20260808 14:29:18.665730 16027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "038e03baa28e490b998cd9af64dcd9f8" candidate_uuid: "96c5dbfefc7c421d8e7d380fee426780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b06e804fc5f4b59b12f27a41b46e807"
I20260808 14:29:18.665799 15735 raft_consensus.cc:3037] T 038e03baa28e490b998cd9af64dcd9f8 P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:18.665884 16027 raft_consensus.cc:3037] T 038e03baa28e490b998cd9af64dcd9f8 P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:18.666730 15735 raft_consensus.cc:2454] T 038e03baa28e490b998cd9af64dcd9f8 P e9170b9c02f3427599c7f9ffe62a370b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96c5dbfefc7c421d8e7d380fee426780 in term 1.
I20260808 14:29:18.666968 16027 raft_consensus.cc:2454] T 038e03baa28e490b998cd9af64dcd9f8 P 6b06e804fc5f4b59b12f27a41b46e807 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96c5dbfefc7c421d8e7d380fee426780 in term 1.
I20260808 14:29:18.667063 15812 leader_election.cc:302] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96c5dbfefc7c421d8e7d380fee426780, e9170b9c02f3427599c7f9ffe62a370b; no voters:
I20260808 14:29:18.667271 16112 raft_consensus.cc:2781] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:29:18.667388 16112 raft_consensus.cc:686] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780 [term 1 LEADER]: Becoming Leader. State: Replica: 96c5dbfefc7c421d8e7d380fee426780, State: Running, Role: LEADER
I20260808 14:29:18.667568 16112 consensus_queue.cc:237] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.668236 16136 tablet_replica.cc:439] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780: TxnStatusTablet state changed. Reason: New leader 96c5dbfefc7c421d8e7d380fee426780. Latest consensus state: current_term: 1 leader_uuid: "96c5dbfefc7c421d8e7d380fee426780" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } } }
I20260808 14:29:18.668362 16136 tablet_replica.cc:442] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780: This TxnStatusTablet replica's current role is: LEADER
I20260808 14:29:18.668702 16148 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 14:29:18.668892 16148 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 14:29:18.669436 15570 catalog_manager.cc:5495] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96c5dbfefc7c421d8e7d380fee426780 (127.7.125.66). New cstate: current_term: 1 leader_uuid: "96c5dbfefc7c421d8e7d380fee426780" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } health_report { overall_health: UNKNOWN } } }
I20260808 14:29:18.689968 16143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:18.685180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:35199 (local address 127.0.0.1:48782)
0808 14:29:18.685621 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:18.685643 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:18.685729 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:18.686234 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:18.686237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:18.686245 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 14:29:18.686361 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:18.686367 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.687362 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.687367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:18.688138 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:18.688149 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.688670 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.688674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:18.688770 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:18.688824 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 14:29:18.689640 (+ 816us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 14:29:18.689658 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 14:29:18.689750 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 14:29:18.689853 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284}
I20260808 14:29:18.689985 16151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:18.685250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48782 (local address 127.7.125.66:35199)
0808 14:29:18.685960 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:18.685966 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:18.685977 (+ 11us) server_negotiation.cc:407] Connection header received
0808 14:29:18.686031 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:18.686033 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:18.686097 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:18.686218 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:29:18.686470 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.687239 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.688274 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.688541 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.688656 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:18.688982 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:29:18.689529 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:29:18.689655 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:18.689762 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 14:29:18.689820 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":116,"threads_started":1}
I20260808 14:29:18.692960 15735 raft_consensus.cc:1261] T 038e03baa28e490b998cd9af64dcd9f8 P e9170b9c02f3427599c7f9ffe62a370b [term 1 FOLLOWER]: Refusing update from remote peer 96c5dbfefc7c421d8e7d380fee426780: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:29:18.693152 16027 raft_consensus.cc:1261] T 038e03baa28e490b998cd9af64dcd9f8 P 6b06e804fc5f4b59b12f27a41b46e807 [term 1 FOLLOWER]: Refusing update from remote peer 96c5dbfefc7c421d8e7d380fee426780: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:29:18.693508 16112 consensus_queue.cc:1035] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:29:18.693682 16136 consensus_queue.cc:1035] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:29:18.695613 16116 tablet_replica.cc:439] T 038e03baa28e490b998cd9af64dcd9f8 P 6b06e804fc5f4b59b12f27a41b46e807: TxnStatusTablet state changed. Reason: New leader 96c5dbfefc7c421d8e7d380fee426780. Latest consensus state: current_term: 1 leader_uuid: "96c5dbfefc7c421d8e7d380fee426780" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } } }
I20260808 14:29:18.695732 16116 tablet_replica.cc:442] T 038e03baa28e490b998cd9af64dcd9f8 P 6b06e804fc5f4b59b12f27a41b46e807: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:29:18.695832 16114 tablet_replica.cc:439] T 038e03baa28e490b998cd9af64dcd9f8 P e9170b9c02f3427599c7f9ffe62a370b: TxnStatusTablet state changed. Reason: New leader 96c5dbfefc7c421d8e7d380fee426780. Latest consensus state: current_term: 1 leader_uuid: "96c5dbfefc7c421d8e7d380fee426780" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } } }
I20260808 14:29:18.695956 16114 tablet_replica.cc:442] T 038e03baa28e490b998cd9af64dcd9f8 P e9170b9c02f3427599c7f9ffe62a370b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:29:18.700934 16136 tablet_replica.cc:439] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "96c5dbfefc7c421d8e7d380fee426780" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } } }
I20260808 14:29:18.701066 16136 tablet_replica.cc:442] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780: This TxnStatusTablet replica's current role is: LEADER
I20260808 14:29:18.701390 16136 tablet_replica.cc:439] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "96c5dbfefc7c421d8e7d380fee426780" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } } }
I20260808 14:29:18.701484 16136 tablet_replica.cc:442] T 038e03baa28e490b998cd9af64dcd9f8 P 96c5dbfefc7c421d8e7d380fee426780: This TxnStatusTablet replica's current role is: LEADER
I20260808 14:29:18.701449 16114 tablet_replica.cc:439] T 038e03baa28e490b998cd9af64dcd9f8 P e9170b9c02f3427599c7f9ffe62a370b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "96c5dbfefc7c421d8e7d380fee426780" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } } }
I20260808 14:29:18.701587 16114 tablet_replica.cc:442] T 038e03baa28e490b998cd9af64dcd9f8 P e9170b9c02f3427599c7f9ffe62a370b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:29:18.704036 16116 tablet_replica.cc:439] T 038e03baa28e490b998cd9af64dcd9f8 P 6b06e804fc5f4b59b12f27a41b46e807: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "96c5dbfefc7c421d8e7d380fee426780" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } } }
I20260808 14:29:18.704166 16116 tablet_replica.cc:442] T 038e03baa28e490b998cd9af64dcd9f8 P 6b06e804fc5f4b59b12f27a41b46e807: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:29:18.710690 16151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:18.707071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48794 (local address 127.7.125.66:35199)
0808 14:29:18.707199 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:18.707204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:18.707221 (+ 17us) server_negotiation.cc:407] Connection header received
0808 14:29:18.707274 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:18.707278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:18.707329 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:18.707427 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:29:18.707764 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.708469 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.709529 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.709781 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.709907 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:18.710186 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:29:18.710274 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:29:18.710360 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:18.710470 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 14:29:18.710530 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 14:29:18.723197 16151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:18.718864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:53607 (local address 127.7.125.66:35199)
0808 14:29:18.719002 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:18.719006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:18.719228 (+ 222us) server_negotiation.cc:407] Connection header received
0808 14:29:18.719329 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:18.719331 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:18.719376 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:18.719453 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:29:18.719803 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.720387 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.721832 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.722168 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.722306 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:18.722650 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:29:18.722729 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:29:18.722795 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:18.722946 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 14:29:18.723068 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":40}
I20260808 14:29:18.723239 16158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:18.718765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:35199 (local address 127.7.125.66:53607)
0808 14:29:18.719102 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:18.719118 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:18.719244 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:18.719488 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:18.719491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:18.719499 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 14:29:18.719668 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:18.719674 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.720507 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.720510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:18.721543 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:18.721554 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.722300 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.722304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:18.722426 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:18.722487 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 14:29:18.722846 (+ 359us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 14:29:18.722853 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 14:29:18.722922 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 14:29:18.723027 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":22,"thread_start_us":112,"threads_started":1}
I20260808 14:29:18.730394 15735 raft_consensus.cc:1261] T 8f941147637a427c8c6119bfa15018d0 P e9170b9c02f3427599c7f9ffe62a370b [term 1 FOLLOWER]: Refusing update from remote peer 96c5dbfefc7c421d8e7d380fee426780: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:29:18.730394 16027 raft_consensus.cc:1261] T 8f941147637a427c8c6119bfa15018d0 P 6b06e804fc5f4b59b12f27a41b46e807 [term 1 FOLLOWER]: Refusing update from remote peer 96c5dbfefc7c421d8e7d380fee426780: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:29:18.730818 16136 consensus_queue.cc:1035] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:29:18.731032 16159 consensus_queue.cc:1035] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:29:18.755861 16122 mvcc.cc:205] Tried to move back new op lower bound from 7316272573356564480 to 7316272571919908864. Current Snapshot: MvccSnapshot[applied={T|T < 7316272573356564480}]
I20260808 14:29:18.772646 15569 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:29:18.772833 15569 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:29:18.775025 15569 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe: Sending DeleteTablet for 3 replicas of tablet 8f941147637a427c8c6119bfa15018d0
I20260808 14:29:18.775640 15714 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f941147637a427c8c6119bfa15018d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:18 UTC) from {username='oryx', principal='oryx/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:47010
I20260808 14:29:18.775669 15862 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f941147637a427c8c6119bfa15018d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:18 UTC) from {username='oryx', principal='oryx/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:48772
I20260808 14:29:18.775892 16007 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f941147637a427c8c6119bfa15018d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:18 UTC) from {username='oryx', principal='oryx/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:35268
I20260808 14:29:18.776250 16167 tablet_replica.cc:330] T 8f941147637a427c8c6119bfa15018d0 P 6b06e804fc5f4b59b12f27a41b46e807: stopping tablet replica
I20260808 14:29:18.776275 16164 tablet_replica.cc:330] T 8f941147637a427c8c6119bfa15018d0 P e9170b9c02f3427599c7f9ffe62a370b: stopping tablet replica
I20260808 14:29:18.776400 16164 raft_consensus.cc:2229] T 8f941147637a427c8c6119bfa15018d0 P e9170b9c02f3427599c7f9ffe62a370b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:29:18.776527 16164 raft_consensus.cc:2258] T 8f941147637a427c8c6119bfa15018d0 P e9170b9c02f3427599c7f9ffe62a370b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:18.777410 16164 ts_tablet_manager.cc:1905] T 8f941147637a427c8c6119bfa15018d0 P e9170b9c02f3427599c7f9ffe62a370b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:29:18.780663 16167 raft_consensus.cc:2229] T 8f941147637a427c8c6119bfa15018d0 P 6b06e804fc5f4b59b12f27a41b46e807 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:29:18.780845 16167 raft_consensus.cc:2258] T 8f941147637a427c8c6119bfa15018d0 P 6b06e804fc5f4b59b12f27a41b46e807 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:18.781236 16164 ts_tablet_manager.cc:1918] T 8f941147637a427c8c6119bfa15018d0 P e9170b9c02f3427599c7f9ffe62a370b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:29:18.781392 16164 log.cc:1196] T 8f941147637a427c8c6119bfa15018d0 P e9170b9c02f3427599c7f9ffe62a370b: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-0/wal/wals/8f941147637a427c8c6119bfa15018d0
I20260808 14:29:18.781610 16167 ts_tablet_manager.cc:1905] T 8f941147637a427c8c6119bfa15018d0 P 6b06e804fc5f4b59b12f27a41b46e807: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:29:18.781854 16164 ts_tablet_manager.cc:1939] T 8f941147637a427c8c6119bfa15018d0 P e9170b9c02f3427599c7f9ffe62a370b: Deleting consensus metadata
I20260808 14:29:18.782959 15556 catalog_manager.cc:4841] TS e9170b9c02f3427599c7f9ffe62a370b (127.7.125.65:34197): tablet 8f941147637a427c8c6119bfa15018d0 (table test-table [id=cceab87ff57449528777f9133218bdc6]) successfully deleted
I20260808 14:29:18.784191 16166 tablet_replica.cc:330] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780: stopping tablet replica
I20260808 14:29:18.784323 16166 raft_consensus.cc:2229] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:29:18.784644 16166 raft_consensus.cc:2258] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:18.784713 16167 ts_tablet_manager.cc:1918] T 8f941147637a427c8c6119bfa15018d0 P 6b06e804fc5f4b59b12f27a41b46e807: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:29:18.784806 16167 log.cc:1196] T 8f941147637a427c8c6119bfa15018d0 P 6b06e804fc5f4b59b12f27a41b46e807: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/8f941147637a427c8c6119bfa15018d0
I20260808 14:29:18.785142 16167 ts_tablet_manager.cc:1939] T 8f941147637a427c8c6119bfa15018d0 P 6b06e804fc5f4b59b12f27a41b46e807: Deleting consensus metadata
I20260808 14:29:18.785313 16166 ts_tablet_manager.cc:1905] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:29:18.785900 15553 catalog_manager.cc:4841] TS 6b06e804fc5f4b59b12f27a41b46e807 (127.7.125.67:44167): tablet 8f941147637a427c8c6119bfa15018d0 (table test-table [id=cceab87ff57449528777f9133218bdc6]) successfully deleted
I20260808 14:29:18.788200 16166 ts_tablet_manager.cc:1918] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:29:18.788283 16166 log.cc:1196] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/8f941147637a427c8c6119bfa15018d0
I20260808 14:29:18.788542 16166 ts_tablet_manager.cc:1939] T 8f941147637a427c8c6119bfa15018d0 P 96c5dbfefc7c421d8e7d380fee426780: Deleting consensus metadata
I20260808 14:29:18.789728 15554 catalog_manager.cc:4841] TS 96c5dbfefc7c421d8e7d380fee426780 (127.7.125.66:35199): tablet 8f941147637a427c8c6119bfa15018d0 (table test-table [id=cceab87ff57449528777f9133218bdc6]) successfully deleted
Aug 08 14:29:18 dist-test-slave-w206 krb5kdc[15524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 14:29:18 dist-test-slave-w206 krb5kdc[15524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:18 dist-test-slave-w206 krb5kdc[15524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 14:29:18.809212 15620 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:18.801751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35658 (local address 127.7.125.126:44075)
0808 14:29:18.801900 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:18.801905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:18.802056 (+ 151us) server_negotiation.cc:407] Connection header received
0808 14:29:18.802205 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:18.802207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:18.802254 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:18.802327 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:18.803366 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.803898 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.805151 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.805367 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.805460 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:18.809083 (+ 3623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35658: BlockingRecv error: recv got EOF from 127.0.0.1:35658 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 14:29:18 dist-test-slave-w206 krb5kdc[15524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM
I20260808 14:29:18.820977 15620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:18.811232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35662 (local address 127.7.125.126:44075)
0808 14:29:18.811380 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:18.811385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:18.811640 (+ 255us) server_negotiation.cc:407] Connection header received
0808 14:29:18.811817 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:18.811820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:18.811876 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:18.811979 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:18.812821 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.813440 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.814562 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:18.814773 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:18.814855 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:18.817683 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:18.817686 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:18.817688 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:18.817714 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:18.819271 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:18.819735 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:18.819737 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:18.819739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:18.819812 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:18.820113 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:18.820115 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:18.820117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:18.820313 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:18.820429 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:18.820710 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 14:29:18.820811 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 14:29:18.823887 15569 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:29:18.824401 15569 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:29:18.829560 15714 tablet_service.cc:1467] Processing CreateTablet for tablet 262b6a8d987c461d81d082b48b908e04 (DEFAULT_TABLE table=test-table [id=762db6827ad246ab9bbd2c4130537c74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:18.829679 15862 tablet_service.cc:1467] Processing CreateTablet for tablet 262b6a8d987c461d81d082b48b908e04 (DEFAULT_TABLE table=test-table [id=762db6827ad246ab9bbd2c4130537c74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:18.829931 15714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 262b6a8d987c461d81d082b48b908e04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:18.830021 15862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 262b6a8d987c461d81d082b48b908e04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:18.829970 16007 tablet_service.cc:1467] Processing CreateTablet for tablet 262b6a8d987c461d81d082b48b908e04 (DEFAULT_TABLE table=test-table [id=762db6827ad246ab9bbd2c4130537c74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:18.830237 16007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 262b6a8d987c461d81d082b48b908e04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:18.832679 16106 tablet_bootstrap.cc:492] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b: Bootstrap starting.
I20260808 14:29:18.832742 16108 tablet_bootstrap.cc:492] T 262b6a8d987c461d81d082b48b908e04 P 6b06e804fc5f4b59b12f27a41b46e807: Bootstrap starting.
I20260808 14:29:18.833683 16106 tablet_bootstrap.cc:654] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:18.833681 16108 tablet_bootstrap.cc:654] T 262b6a8d987c461d81d082b48b908e04 P 6b06e804fc5f4b59b12f27a41b46e807: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:18.834687 16105 tablet_bootstrap.cc:492] T 262b6a8d987c461d81d082b48b908e04 P 96c5dbfefc7c421d8e7d380fee426780: Bootstrap starting.
I20260808 14:29:18.835347 16108 tablet_bootstrap.cc:492] T 262b6a8d987c461d81d082b48b908e04 P 6b06e804fc5f4b59b12f27a41b46e807: No bootstrap required, opened a new log
I20260808 14:29:18.835448 16108 ts_tablet_manager.cc:1397] T 262b6a8d987c461d81d082b48b908e04 P 6b06e804fc5f4b59b12f27a41b46e807: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 14:29:18.835734 16105 tablet_bootstrap.cc:654] T 262b6a8d987c461d81d082b48b908e04 P 96c5dbfefc7c421d8e7d380fee426780: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:18.835918 16108 raft_consensus.cc:348] T 262b6a8d987c461d81d082b48b908e04 P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.836047 16108 raft_consensus.cc:374] T 262b6a8d987c461d81d082b48b908e04 P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:18.836092 16108 raft_consensus.cc:729] T 262b6a8d987c461d81d082b48b908e04 P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b06e804fc5f4b59b12f27a41b46e807, State: Initialized, Role: FOLLOWER
I20260808 14:29:18.836225 16108 consensus_queue.cc:260] T 262b6a8d987c461d81d082b48b908e04 P 6b06e804fc5f4b59b12f27a41b46e807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.836884 16108 ts_tablet_manager.cc:1428] T 262b6a8d987c461d81d082b48b908e04 P 6b06e804fc5f4b59b12f27a41b46e807: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 14:29:18.837180 16105 tablet_bootstrap.cc:492] T 262b6a8d987c461d81d082b48b908e04 P 96c5dbfefc7c421d8e7d380fee426780: No bootstrap required, opened a new log
I20260808 14:29:18.837267 16105 ts_tablet_manager.cc:1397] T 262b6a8d987c461d81d082b48b908e04 P 96c5dbfefc7c421d8e7d380fee426780: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:29:18.837678 16105 raft_consensus.cc:348] T 262b6a8d987c461d81d082b48b908e04 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.837827 16105 raft_consensus.cc:374] T 262b6a8d987c461d81d082b48b908e04 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:18.837862 16105 raft_consensus.cc:729] T 262b6a8d987c461d81d082b48b908e04 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96c5dbfefc7c421d8e7d380fee426780, State: Initialized, Role: FOLLOWER
I20260808 14:29:18.838035 16105 consensus_queue.cc:260] T 262b6a8d987c461d81d082b48b908e04 P 96c5dbfefc7c421d8e7d380fee426780 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.838305 16105 ts_tablet_manager.cc:1428] T 262b6a8d987c461d81d082b48b908e04 P 96c5dbfefc7c421d8e7d380fee426780: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:29:18.839634 16106 tablet_bootstrap.cc:492] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b: No bootstrap required, opened a new log
I20260808 14:29:18.839747 16106 ts_tablet_manager.cc:1397] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260808 14:29:18.840547 16106 raft_consensus.cc:348] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.840646 16106 raft_consensus.cc:374] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:18.840667 16106 raft_consensus.cc:729] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9170b9c02f3427599c7f9ffe62a370b, State: Initialized, Role: FOLLOWER
I20260808 14:29:18.840829 16106 consensus_queue.cc:260] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:18.841127 16106 ts_tablet_manager.cc:1428] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 14:29:19.123828 16114 raft_consensus.cc:482] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:29:19.123994 16114 raft_consensus.cc:504] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:19.125077 16114 leader_election.cc:288] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96c5dbfefc7c421d8e7d380fee426780 (127.7.125.66:35199), 6b06e804fc5f4b59b12f27a41b46e807 (127.7.125.67:44167)
I20260808 14:29:19.129179 16183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:19.125324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:35199 (local address 127.7.125.65:49615)
0808 14:29:19.125639 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:19.125658 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:19.125772 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:19.126078 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:19.126081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:19.126097 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:19.126228 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:19.126235 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:19.127107 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:19.127110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:19.128052 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:19.128061 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:19.128739 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:19.128743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:19.128825 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:19.128851 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 14:29:19.128928 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 14:29:19.128984 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":6,"thread_start_us":99,"threads_started":1}
I20260808 14:29:19.129204 16151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:19.125473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:49615 (local address 127.7.125.66:35199)
0808 14:29:19.125719 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:19.125726 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:19.125749 (+ 23us) server_negotiation.cc:407] Connection header received
0808 14:29:19.125905 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:19.125911 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:19.125971 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:19.126040 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:19.126349 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:19.126933 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:19.128181 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:19.128617 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:19.128736 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:19.128794 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:19.128966 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 14:29:19.129076 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157}
I20260808 14:29:19.129268 16185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:19.125722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:56535 (local address 127.7.125.67:44167)
0808 14:29:19.125981 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:19.125984 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:19.125996 (+ 12us) server_negotiation.cc:407] Connection header received
0808 14:29:19.126065 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:19.126068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:19.126116 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:19.126203 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:19.126543 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:19.127116 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:19.128181 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:19.128638 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:19.128777 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:19.128837 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:19.129053 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 14:29:19.129127 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":93,"threads_started":1}
I20260808 14:29:19.129220 16184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:19.125371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:44167 (local address 127.7.125.65:56535)
0808 14:29:19.125716 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:19.125730 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:19.125859 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:19.126246 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:19.126248 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:19.126263 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:19.126415 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:19.126423 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:19.127217 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:19.127220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:19.128048 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:19.128057 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:19.128784 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:19.128788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:19.128887 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:19.128938 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 14:29:19.129017 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 14:29:19.129084 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":23,"thread_start_us":81,"threads_started":1}
I20260808 14:29:19.129881 15884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "262b6a8d987c461d81d082b48b908e04" candidate_uuid: "e9170b9c02f3427599c7f9ffe62a370b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96c5dbfefc7c421d8e7d380fee426780" is_pre_election: true
I20260808 14:29:19.129879 16027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "262b6a8d987c461d81d082b48b908e04" candidate_uuid: "e9170b9c02f3427599c7f9ffe62a370b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b06e804fc5f4b59b12f27a41b46e807" is_pre_election: true
I20260808 14:29:19.130048 15884 raft_consensus.cc:2454] T 262b6a8d987c461d81d082b48b908e04 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9170b9c02f3427599c7f9ffe62a370b in term 0.
I20260808 14:29:19.130060 16027 raft_consensus.cc:2454] T 262b6a8d987c461d81d082b48b908e04 P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9170b9c02f3427599c7f9ffe62a370b in term 0.
I20260808 14:29:19.130460 15661 leader_election.cc:302] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b06e804fc5f4b59b12f27a41b46e807, e9170b9c02f3427599c7f9ffe62a370b; no voters:
I20260808 14:29:19.130723 16114 raft_consensus.cc:2781] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:29:19.130798 16114 raft_consensus.cc:482] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:29:19.130852 16114 raft_consensus.cc:3037] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:19.131939 16114 raft_consensus.cc:504] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:19.132288 16114 leader_election.cc:288] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b [CANDIDATE]: Term 1 election: Requested vote from peers 96c5dbfefc7c421d8e7d380fee426780 (127.7.125.66:35199), 6b06e804fc5f4b59b12f27a41b46e807 (127.7.125.67:44167)
I20260808 14:29:19.132670 15884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "262b6a8d987c461d81d082b48b908e04" candidate_uuid: "e9170b9c02f3427599c7f9ffe62a370b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96c5dbfefc7c421d8e7d380fee426780"
I20260808 14:29:19.132670 16027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "262b6a8d987c461d81d082b48b908e04" candidate_uuid: "e9170b9c02f3427599c7f9ffe62a370b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b06e804fc5f4b59b12f27a41b46e807"
I20260808 14:29:19.132798 15884 raft_consensus.cc:3037] T 262b6a8d987c461d81d082b48b908e04 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:19.132798 16027 raft_consensus.cc:3037] T 262b6a8d987c461d81d082b48b908e04 P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:19.133929 15884 raft_consensus.cc:2454] T 262b6a8d987c461d81d082b48b908e04 P 96c5dbfefc7c421d8e7d380fee426780 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9170b9c02f3427599c7f9ffe62a370b in term 1.
I20260808 14:29:19.133929 16027 raft_consensus.cc:2454] T 262b6a8d987c461d81d082b48b908e04 P 6b06e804fc5f4b59b12f27a41b46e807 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9170b9c02f3427599c7f9ffe62a370b in term 1.
I20260808 14:29:19.134198 15661 leader_election.cc:302] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b06e804fc5f4b59b12f27a41b46e807, e9170b9c02f3427599c7f9ffe62a370b; no voters:
I20260808 14:29:19.134370 16114 raft_consensus.cc:2781] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:29:19.134567 16114 raft_consensus.cc:686] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b [term 1 LEADER]: Becoming Leader. State: Replica: e9170b9c02f3427599c7f9ffe62a370b, State: Running, Role: LEADER
I20260808 14:29:19.134827 16114 consensus_queue.cc:237] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:19.137336 15570 catalog_manager.cc:5495] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b reported cstate change: term changed from 0 to 1, leader changed from <none> to e9170b9c02f3427599c7f9ffe62a370b (127.7.125.65). New cstate: current_term: 1 leader_uuid: "e9170b9c02f3427599c7f9ffe62a370b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } health_report { overall_health: UNKNOWN } } }
I20260808 14:29:19.210321 16189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:19.206303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47018 (local address 127.7.125.65:34197)
0808 14:29:19.206625 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:19.206632 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:19.206645 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:29:19.206731 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:19.206733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:19.206788 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:19.206938 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:29:19.207225 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:19.207796 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:19.208742 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:19.208998 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:19.209137 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:19.209445 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:29:19.209909 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:29:19.210042 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:19.210106 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 14:29:19.210159 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":102,"threads_started":1}
I20260808 14:29:19.213142 16027 raft_consensus.cc:1261] T 262b6a8d987c461d81d082b48b908e04 P 6b06e804fc5f4b59b12f27a41b46e807 [term 1 FOLLOWER]: Refusing update from remote peer e9170b9c02f3427599c7f9ffe62a370b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:29:19.213171 15884 raft_consensus.cc:1261] T 262b6a8d987c461d81d082b48b908e04 P 96c5dbfefc7c421d8e7d380fee426780 [term 1 FOLLOWER]: Refusing update from remote peer e9170b9c02f3427599c7f9ffe62a370b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:29:19.213641 16186 consensus_queue.cc:1035] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:29:19.213898 16114 consensus_queue.cc:1035] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b [LEADER]: Connected to new peer: Peer: permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:29:19.224570 15569 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:29:19.224741 15569 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:29:19.226784 15569 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe: Sending DeleteTablet for 3 replicas of tablet 262b6a8d987c461d81d082b48b908e04
I20260808 14:29:19.227389 15714 tablet_service.cc:1514] Processing DeleteTablet for tablet 262b6a8d987c461d81d082b48b908e04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:19 UTC) from {username='oryx', principal='oryx/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:47010
I20260808 14:29:19.227463 15862 tablet_service.cc:1514] Processing DeleteTablet for tablet 262b6a8d987c461d81d082b48b908e04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:19 UTC) from {username='oryx', principal='oryx/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:48772
I20260808 14:29:19.227571 16164 tablet_replica.cc:330] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b: stopping tablet replica
I20260808 14:29:19.227586 16007 tablet_service.cc:1514] Processing DeleteTablet for tablet 262b6a8d987c461d81d082b48b908e04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:19 UTC) from {username='oryx', principal='oryx/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:35268
I20260808 14:29:19.227638 16166 tablet_replica.cc:330] T 262b6a8d987c461d81d082b48b908e04 P 96c5dbfefc7c421d8e7d380fee426780: stopping tablet replica
I20260808 14:29:19.227690 16164 raft_consensus.cc:2229] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:29:19.227743 16166 raft_consensus.cc:2229] T 262b6a8d987c461d81d082b48b908e04 P 96c5dbfefc7c421d8e7d380fee426780 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:29:19.227887 16166 raft_consensus.cc:2258] T 262b6a8d987c461d81d082b48b908e04 P 96c5dbfefc7c421d8e7d380fee426780 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:19.227938 16164 raft_consensus.cc:2258] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:19.228061 16167 tablet_replica.cc:330] T 262b6a8d987c461d81d082b48b908e04 P 6b06e804fc5f4b59b12f27a41b46e807: stopping tablet replica
I20260808 14:29:19.228170 16167 raft_consensus.cc:2229] T 262b6a8d987c461d81d082b48b908e04 P 6b06e804fc5f4b59b12f27a41b46e807 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:29:19.228297 16167 raft_consensus.cc:2258] T 262b6a8d987c461d81d082b48b908e04 P 6b06e804fc5f4b59b12f27a41b46e807 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:19.228984 15569 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 14:29:19.229221 16167 ts_tablet_manager.cc:1905] T 262b6a8d987c461d81d082b48b908e04 P 6b06e804fc5f4b59b12f27a41b46e807: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 14:29:19.229406 15569 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:29:19.231868 16167 ts_tablet_manager.cc:1918] T 262b6a8d987c461d81d082b48b908e04 P 6b06e804fc5f4b59b12f27a41b46e807: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:29:19.231945 16167 log.cc:1196] T 262b6a8d987c461d81d082b48b908e04 P 6b06e804fc5f4b59b12f27a41b46e807: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/262b6a8d987c461d81d082b48b908e04
I20260808 14:29:19.232213 16167 ts_tablet_manager.cc:1939] T 262b6a8d987c461d81d082b48b908e04 P 6b06e804fc5f4b59b12f27a41b46e807: Deleting consensus metadata
I20260808 14:29:19.233198 16166 ts_tablet_manager.cc:1905] T 262b6a8d987c461d81d082b48b908e04 P 96c5dbfefc7c421d8e7d380fee426780: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:29:19.233410 15553 catalog_manager.cc:4841] TS 6b06e804fc5f4b59b12f27a41b46e807 (127.7.125.67:44167): tablet 262b6a8d987c461d81d082b48b908e04 (table test-table [id=762db6827ad246ab9bbd2c4130537c74]) successfully deleted
I20260808 14:29:19.236013 16166 ts_tablet_manager.cc:1918] T 262b6a8d987c461d81d082b48b908e04 P 96c5dbfefc7c421d8e7d380fee426780: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:29:19.236104 16166 log.cc:1196] T 262b6a8d987c461d81d082b48b908e04 P 96c5dbfefc7c421d8e7d380fee426780: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/262b6a8d987c461d81d082b48b908e04
I20260808 14:29:19.237098 15862 tablet_service.cc:1467] Processing CreateTablet for tablet 823084c94a204833b341bd929b1f4398 (DEFAULT_TABLE table=test-table [id=ae064552786742368f6a3f9016370abf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:19.237355 15862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 823084c94a204833b341bd929b1f4398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:19.239660 16105 tablet_bootstrap.cc:492] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780: Bootstrap starting.
I20260808 14:29:19.239615 15714 tablet_service.cc:1467] Processing CreateTablet for tablet 823084c94a204833b341bd929b1f4398 (DEFAULT_TABLE table=test-table [id=ae064552786742368f6a3f9016370abf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:19.239951 15714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 823084c94a204833b341bd929b1f4398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:19.240491 16105 tablet_bootstrap.cc:654] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:19.241288 16007 tablet_service.cc:1467] Processing CreateTablet for tablet 823084c94a204833b341bd929b1f4398 (DEFAULT_TABLE table=test-table [id=ae064552786742368f6a3f9016370abf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:19.241539 16007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 823084c94a204833b341bd929b1f4398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:19.242617 16106 tablet_bootstrap.cc:492] T 823084c94a204833b341bd929b1f4398 P e9170b9c02f3427599c7f9ffe62a370b: Bootstrap starting.
I20260808 14:29:19.243683 16106 tablet_bootstrap.cc:654] T 823084c94a204833b341bd929b1f4398 P e9170b9c02f3427599c7f9ffe62a370b: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:19.243791 16108 tablet_bootstrap.cc:492] T 823084c94a204833b341bd929b1f4398 P 6b06e804fc5f4b59b12f27a41b46e807: Bootstrap starting.
I20260808 14:29:19.244639 16108 tablet_bootstrap.cc:654] T 823084c94a204833b341bd929b1f4398 P 6b06e804fc5f4b59b12f27a41b46e807: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:19.244652 16105 tablet_bootstrap.cc:492] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780: No bootstrap required, opened a new log
I20260808 14:29:19.244728 16105 ts_tablet_manager.cc:1397] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260808 14:29:19.245127 16105 raft_consensus.cc:348] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:19.245246 16105 raft_consensus.cc:374] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:19.245280 16105 raft_consensus.cc:729] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96c5dbfefc7c421d8e7d380fee426780, State: Initialized, Role: FOLLOWER
I20260808 14:29:19.245395 16105 consensus_queue.cc:260] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:19.245669 16105 ts_tablet_manager.cc:1428] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:29:19.246168 16164 ts_tablet_manager.cc:1905] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:29:19.246261 16106 tablet_bootstrap.cc:492] T 823084c94a204833b341bd929b1f4398 P e9170b9c02f3427599c7f9ffe62a370b: No bootstrap required, opened a new log
I20260808 14:29:19.246344 16106 ts_tablet_manager.cc:1397] T 823084c94a204833b341bd929b1f4398 P e9170b9c02f3427599c7f9ffe62a370b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 14:29:19.246783 16106 raft_consensus.cc:348] T 823084c94a204833b341bd929b1f4398 P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:19.246915 16106 raft_consensus.cc:374] T 823084c94a204833b341bd929b1f4398 P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:19.246958 16106 raft_consensus.cc:729] T 823084c94a204833b341bd929b1f4398 P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9170b9c02f3427599c7f9ffe62a370b, State: Initialized, Role: FOLLOWER
I20260808 14:29:19.247113 16106 consensus_queue.cc:260] T 823084c94a204833b341bd929b1f4398 P e9170b9c02f3427599c7f9ffe62a370b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:19.247380 16106 ts_tablet_manager.cc:1428] T 823084c94a204833b341bd929b1f4398 P e9170b9c02f3427599c7f9ffe62a370b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:29:19.248416 16164 ts_tablet_manager.cc:1918] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:29:19.248492 16164 log.cc:1196] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-0/wal/wals/262b6a8d987c461d81d082b48b908e04
I20260808 14:29:19.248741 16164 ts_tablet_manager.cc:1939] T 262b6a8d987c461d81d082b48b908e04 P e9170b9c02f3427599c7f9ffe62a370b: Deleting consensus metadata
I20260808 14:29:19.249699 15556 catalog_manager.cc:4841] TS e9170b9c02f3427599c7f9ffe62a370b (127.7.125.65:34197): tablet 262b6a8d987c461d81d082b48b908e04 (table test-table [id=762db6827ad246ab9bbd2c4130537c74]) successfully deleted
I20260808 14:29:19.250387 16166 ts_tablet_manager.cc:1939] T 262b6a8d987c461d81d082b48b908e04 P 96c5dbfefc7c421d8e7d380fee426780: Deleting consensus metadata
I20260808 14:29:19.251600 15554 catalog_manager.cc:4841] TS 96c5dbfefc7c421d8e7d380fee426780 (127.7.125.66:35199): tablet 262b6a8d987c461d81d082b48b908e04 (table test-table [id=762db6827ad246ab9bbd2c4130537c74]) successfully deleted
I20260808 14:29:19.252087 16108 tablet_bootstrap.cc:492] T 823084c94a204833b341bd929b1f4398 P 6b06e804fc5f4b59b12f27a41b46e807: No bootstrap required, opened a new log
I20260808 14:29:19.252173 16108 ts_tablet_manager.cc:1397] T 823084c94a204833b341bd929b1f4398 P 6b06e804fc5f4b59b12f27a41b46e807: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260808 14:29:19.252560 16108 raft_consensus.cc:348] T 823084c94a204833b341bd929b1f4398 P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:19.252677 16108 raft_consensus.cc:374] T 823084c94a204833b341bd929b1f4398 P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:19.252708 16108 raft_consensus.cc:729] T 823084c94a204833b341bd929b1f4398 P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b06e804fc5f4b59b12f27a41b46e807, State: Initialized, Role: FOLLOWER
I20260808 14:29:19.252837 16108 consensus_queue.cc:260] T 823084c94a204833b341bd929b1f4398 P 6b06e804fc5f4b59b12f27a41b46e807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:19.253123 16108 ts_tablet_manager.cc:1428] T 823084c94a204833b341bd929b1f4398 P 6b06e804fc5f4b59b12f27a41b46e807: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 14:29:19.254912 16163 raft_consensus.cc:482] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:29:19.255038 16163 raft_consensus.cc:504] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:19.255506 16163 leader_election.cc:288] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9170b9c02f3427599c7f9ffe62a370b (127.7.125.65:34197), 6b06e804fc5f4b59b12f27a41b46e807 (127.7.125.67:44167)
I20260808 14:29:19.255854 15735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "823084c94a204833b341bd929b1f4398" candidate_uuid: "96c5dbfefc7c421d8e7d380fee426780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9170b9c02f3427599c7f9ffe62a370b" is_pre_election: true
I20260808 14:29:19.255946 16027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "823084c94a204833b341bd929b1f4398" candidate_uuid: "96c5dbfefc7c421d8e7d380fee426780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b06e804fc5f4b59b12f27a41b46e807" is_pre_election: true
I20260808 14:29:19.256012 15735 raft_consensus.cc:2454] T 823084c94a204833b341bd929b1f4398 P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96c5dbfefc7c421d8e7d380fee426780 in term 0.
I20260808 14:29:19.256083 16027 raft_consensus.cc:2454] T 823084c94a204833b341bd929b1f4398 P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96c5dbfefc7c421d8e7d380fee426780 in term 0.
I20260808 14:29:19.256371 15812 leader_election.cc:302] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96c5dbfefc7c421d8e7d380fee426780, e9170b9c02f3427599c7f9ffe62a370b; no voters:
I20260808 14:29:19.256583 16163 raft_consensus.cc:2781] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:29:19.256642 16163 raft_consensus.cc:482] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:29:19.256691 16163 raft_consensus.cc:3037] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:19.258041 16163 raft_consensus.cc:504] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:19.258414 16163 leader_election.cc:288] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780 [CANDIDATE]: Term 1 election: Requested vote from peers e9170b9c02f3427599c7f9ffe62a370b (127.7.125.65:34197), 6b06e804fc5f4b59b12f27a41b46e807 (127.7.125.67:44167)
I20260808 14:29:19.258787 16027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "823084c94a204833b341bd929b1f4398" candidate_uuid: "96c5dbfefc7c421d8e7d380fee426780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b06e804fc5f4b59b12f27a41b46e807"
I20260808 14:29:19.258906 16027 raft_consensus.cc:3037] T 823084c94a204833b341bd929b1f4398 P 6b06e804fc5f4b59b12f27a41b46e807 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:19.259971 15735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "823084c94a204833b341bd929b1f4398" candidate_uuid: "96c5dbfefc7c421d8e7d380fee426780" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9170b9c02f3427599c7f9ffe62a370b"
I20260808 14:29:19.260106 15735 raft_consensus.cc:3037] T 823084c94a204833b341bd929b1f4398 P e9170b9c02f3427599c7f9ffe62a370b [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:19.260651 16027 raft_consensus.cc:2454] T 823084c94a204833b341bd929b1f4398 P 6b06e804fc5f4b59b12f27a41b46e807 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96c5dbfefc7c421d8e7d380fee426780 in term 1.
I20260808 14:29:19.260968 15809 leader_election.cc:302] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b06e804fc5f4b59b12f27a41b46e807, 96c5dbfefc7c421d8e7d380fee426780; no voters:
I20260808 14:29:19.261128 16163 raft_consensus.cc:2781] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:29:19.261142 15735 raft_consensus.cc:2454] T 823084c94a204833b341bd929b1f4398 P e9170b9c02f3427599c7f9ffe62a370b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96c5dbfefc7c421d8e7d380fee426780 in term 1.
I20260808 14:29:19.261226 16163 raft_consensus.cc:686] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780 [term 1 LEADER]: Becoming Leader. State: Replica: 96c5dbfefc7c421d8e7d380fee426780, State: Running, Role: LEADER
I20260808 14:29:19.261371 16163 consensus_queue.cc:237] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } }
I20260808 14:29:19.262997 15570 catalog_manager.cc:5495] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96c5dbfefc7c421d8e7d380fee426780 (127.7.125.66). New cstate: current_term: 1 leader_uuid: "96c5dbfefc7c421d8e7d380fee426780" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96c5dbfefc7c421d8e7d380fee426780" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 35199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 } health_report { overall_health: UNKNOWN } } }
I20260808 14:29:19.280260 16151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:19.277058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48804 (local address 127.7.125.66:35199)
0808 14:29:19.277405 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:19.277408 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:19.277418 (+ 10us) server_negotiation.cc:407] Connection header received
0808 14:29:19.277450 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:19.277452 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:19.277487 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:19.277568 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:29:19.277808 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:19.278329 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:19.279225 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:19.279451 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:19.279554 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:19.279811 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:29:19.279880 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:29:19.279970 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:19.280067 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 14:29:19.280118 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282}
I20260808 14:29:19.286442 16027 raft_consensus.cc:1261] T 823084c94a204833b341bd929b1f4398 P 6b06e804fc5f4b59b12f27a41b46e807 [term 1 FOLLOWER]: Refusing update from remote peer 96c5dbfefc7c421d8e7d380fee426780: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:29:19.286465 15735 raft_consensus.cc:1261] T 823084c94a204833b341bd929b1f4398 P e9170b9c02f3427599c7f9ffe62a370b [term 1 FOLLOWER]: Refusing update from remote peer 96c5dbfefc7c421d8e7d380fee426780: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:29:19.286839 16112 consensus_queue.cc:1035] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9170b9c02f3427599c7f9ffe62a370b" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:29:19.287088 16163 consensus_queue.cc:1035] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b06e804fc5f4b59b12f27a41b46e807" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:29:19.321442 15569 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:29:19.321630 15569 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:29:19.323618 15569 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dabd75c6d2d442619e1a65914cec0ebe: Sending DeleteTablet for 3 replicas of tablet 823084c94a204833b341bd929b1f4398
I20260808 14:29:19.324167 15714 tablet_service.cc:1514] Processing DeleteTablet for tablet 823084c94a204833b341bd929b1f4398 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:19 UTC) from {username='oryx', principal='oryx/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:47010
I20260808 14:29:19.324493 16164 tablet_replica.cc:330] T 823084c94a204833b341bd929b1f4398 P e9170b9c02f3427599c7f9ffe62a370b: stopping tablet replica
I20260808 14:29:19.324467 16007 tablet_service.cc:1514] Processing DeleteTablet for tablet 823084c94a204833b341bd929b1f4398 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:19 UTC) from {username='oryx', principal='oryx/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:35268
I20260808 14:29:19.324625 16164 raft_consensus.cc:2229] T 823084c94a204833b341bd929b1f4398 P e9170b9c02f3427599c7f9ffe62a370b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:29:19.324656 16167 tablet_replica.cc:330] T 823084c94a204833b341bd929b1f4398 P 6b06e804fc5f4b59b12f27a41b46e807: stopping tablet replica
I20260808 14:29:19.324734 16164 raft_consensus.cc:2258] T 823084c94a204833b341bd929b1f4398 P e9170b9c02f3427599c7f9ffe62a370b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:19.324767 16167 raft_consensus.cc:2229] T 823084c94a204833b341bd929b1f4398 P 6b06e804fc5f4b59b12f27a41b46e807 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:29:19.324870 16167 raft_consensus.cc:2258] T 823084c94a204833b341bd929b1f4398 P 6b06e804fc5f4b59b12f27a41b46e807 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:19.325186 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 15645
I20260808 14:29:19.325636 16167 ts_tablet_manager.cc:1905] T 823084c94a204833b341bd929b1f4398 P 6b06e804fc5f4b59b12f27a41b46e807: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:29:19.327016 15863 tablet_service.cc:1514] Processing DeleteTablet for tablet 823084c94a204833b341bd929b1f4398 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:19 UTC) from {username='oryx', principal='oryx/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:48772
I20260808 14:29:19.327209 16166 tablet_replica.cc:330] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780: stopping tablet replica
I20260808 14:29:19.327356 16166 raft_consensus.cc:2229] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:29:19.327536 16166 raft_consensus.cc:2258] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:19.328613 16167 ts_tablet_manager.cc:1918] T 823084c94a204833b341bd929b1f4398 P 6b06e804fc5f4b59b12f27a41b46e807: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:29:19.328711 16167 log.cc:1196] T 823084c94a204833b341bd929b1f4398 P 6b06e804fc5f4b59b12f27a41b46e807: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/823084c94a204833b341bd929b1f4398
I20260808 14:29:19.328972 16167 ts_tablet_manager.cc:1939] T 823084c94a204833b341bd929b1f4398 P 6b06e804fc5f4b59b12f27a41b46e807: Deleting consensus metadata
I20260808 14:29:19.329938 15553 catalog_manager.cc:4841] TS 6b06e804fc5f4b59b12f27a41b46e807 (127.7.125.67:44167): tablet 823084c94a204833b341bd929b1f4398 (table test-table [id=ae064552786742368f6a3f9016370abf]) successfully deleted
I20260808 14:29:19.330001 16166 ts_tablet_manager.cc:1905] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:29:19.332080 16166 ts_tablet_manager.cc:1918] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:29:19.332181 16166 log.cc:1196] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/823084c94a204833b341bd929b1f4398
I20260808 14:29:19.332515 16166 ts_tablet_manager.cc:1939] T 823084c94a204833b341bd929b1f4398 P 96c5dbfefc7c421d8e7d380fee426780: Deleting consensus metadata
I20260808 14:29:19.333230 15554 catalog_manager.cc:4841] TS 96c5dbfefc7c421d8e7d380fee426780 (127.7.125.66:35199): tablet 823084c94a204833b341bd929b1f4398 (table test-table [id=ae064552786742368f6a3f9016370abf]) successfully deleted
W20260808 14:29:19.334750 15556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.125.65:34197 (error 108)
I20260808 14:29:19.334954 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 15793
I20260808 14:29:19.337527 16205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:19.337152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:34197 (local address 127.0.0.1:47020)
0808 14:29:19.337375 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:19.337441 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.65:34197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":68,"threads_started":1}
W20260808 14:29:19.337764 15556 catalog_manager.cc:4584] TS e9170b9c02f3427599c7f9ffe62a370b (127.7.125.65:34197): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 823084c94a204833b341bd929b1f4398: Network error: Client connection negotiation failed: client connection to 127.7.125.65:34197: connect: Connection refused (error 111)
I20260808 14:29:19.343921 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 15941
I20260808 14:29:19.351976 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 15537
2026-08-08T14:29:19Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3563 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:29:19 dist-test-slave-w206 krb5kdc[16208](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:29:19 dist-test-slave-w206 krb5kdc[16208](info): set up 2 sockets
Aug 08 14:29:19 dist-test-slave-w206 krb5kdc[16208](info): commencing operation
krb5kdc: starting...
W20260808 14:29:20.512234 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:29:20 dist-test-slave-w206 krb5kdc[16208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:29:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:29:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:29:20.644510 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:37447
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:37447
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:20.756651 16223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:20.756934 16223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:20.756975 16223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:20.761035 16223 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:29:20.761094 16223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:20.761113 16223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:20.761130 16223 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:29:20.761194 16223 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:29:20.766484 16223 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:37447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:37447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:20.767642 16223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:20.768770 16223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:20.777590 16229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:20.777614 16231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:20.777707 16223 server_base.cc:1034] running on GCE node
W20260808 14:29:20.777784 16228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:20.778406 16223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:20.779601 16223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:20.780777 16223 hybrid_clock.cc:648] HybridClock initialized: now 1786199360780745 us; error 44 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T14:29:20Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1438 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:29:20 dist-test-slave-w206 krb5kdc[16236](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:29:20 dist-test-slave-w206 krb5kdc[16236](info): set up 2 sockets
Aug 08 14:29:20 dist-test-slave-w206 krb5kdc[16236](info): commencing operation
krb5kdc: starting...
W20260808 14:29:21.861172 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:29:21 dist-test-slave-w206 krb5kdc[16236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:29:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:29:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:29:21.991549 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:36447
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:36447
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:22.106096 16252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:22.106431 16252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:22.106515 16252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:22.110700 16252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:29:22.110805 16252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:22.110841 16252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:22.110881 16252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:29:22.110929 16252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:29:22.116446 16252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:36447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:36447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:22.117615 16252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:22.118680 16252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:22.127280 16258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:22.127280 16260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:22.127336 16257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:22.127516 16252 server_base.cc:1034] running on GCE node
I20260808 14:29:22.128156 16252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:22.129242 16252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:22.130441 16252 hybrid_clock.cc:648] HybridClock initialized: now 1786199362130427 us; error 24 us; skew 500 ppm
Aug 08 14:29:22 dist-test-slave-w206 krb5kdc[16236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199362, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:22.133229 16252 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:29:22.134188 16252 webserver.cc:460] Webserver started at http://127.7.125.126:42873/ using document root <none> and password file <none>
I20260808 14:29:22.134749 16252 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:22.134842 16252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:22.135135 16252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:22.137193 16252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "c6bb3638781e4fb6a225791e7d23d00e"
format_stamp: "Formatted at 2026-08-08 14:29:22 on dist-test-slave-w206"
server_key: "6a7e26e66be96bfdc07a32b903892a67"
server_key_iv: "65e07bd13f0be13d2587aa5d3d979d4b"
server_key_version: "encryptionkey@0"
I20260808 14:29:22.137786 16252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "c6bb3638781e4fb6a225791e7d23d00e"
format_stamp: "Formatted at 2026-08-08 14:29:22 on dist-test-slave-w206"
server_key: "6a7e26e66be96bfdc07a32b903892a67"
server_key_iv: "65e07bd13f0be13d2587aa5d3d979d4b"
server_key_version: "encryptionkey@0"
I20260808 14:29:22.141670 16252 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260808 14:29:22.144249 16267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:22.145658 16252 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:29:22.145807 16252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "c6bb3638781e4fb6a225791e7d23d00e"
format_stamp: "Formatted at 2026-08-08 14:29:22 on dist-test-slave-w206"
server_key: "6a7e26e66be96bfdc07a32b903892a67"
server_key_iv: "65e07bd13f0be13d2587aa5d3d979d4b"
server_key_version: "encryptionkey@0"
I20260808 14:29:22.145936 16252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:22.163733 16252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:22.167320 16252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:22.167557 16252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:22.175221 16252 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:36447
I20260808 14:29:22.175241 16319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:36447 every 8 connection(s)
I20260808 14:29:22.176235 16252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:29:22.178786 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 16252
I20260808 14:29:22.178949 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:29:22.179232 16320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:22.179337 7669 external_mini_cluster.cc:1428] Setting key 40540ccc41c341d7ea50189329a3004d
I20260808 14:29:22.185799 16320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e: Bootstrap starting.
I20260808 14:29:22.188337 16320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e: Neither blocks nor log segments found. Creating new log.
Aug 08 14:29:22 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:29:22.189275 16320 log.cc:824] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:22.191635 16320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e: No bootstrap required, opened a new log
I20260808 14:29:22.194202 16323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:22.180855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54334 (local address 127.7.125.126:36447)
0808 14:29:22.181330 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:22.181346 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:22.181382 (+ 36us) server_negotiation.cc:407] Connection header received
0808 14:29:22.182136 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:22.182156 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:22.182479 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:22.182830 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:22.183713 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:22.184728 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:22.185601 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:22.186133 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:22.186302 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:22.189280 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:22.189285 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:22.189299 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:22.189326 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:22.191215 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:22.191728 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:22.191738 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:22.191745 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:22.191872 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:22.192187 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:22.192189 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:22.192190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:22.192576 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:22.192726 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:22.193039 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 14:29:22.193399 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":149,"threads_started":1}
I20260808 14:29:22.195885 16320 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bb3638781e4fb6a225791e7d23d00e" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36447 } }
I20260808 14:29:22.196106 16320 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:22.196141 16320 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6bb3638781e4fb6a225791e7d23d00e, State: Initialized, Role: FOLLOWER
I20260808 14:29:22.196564 16320 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bb3638781e4fb6a225791e7d23d00e" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36447 } }
I20260808 14:29:22.196681 16320 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:29:22.196762 16320 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:29:22.196862 16320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:22.197808 16320 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bb3638781e4fb6a225791e7d23d00e" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36447 } }
I20260808 14:29:22.198151 16320 leader_election.cc:302] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6bb3638781e4fb6a225791e7d23d00e; no voters:
I20260808 14:29:22.198501 16320 leader_election.cc:288] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:29:22.198643 16325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:29:22.198896 16325 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [term 1 LEADER]: Becoming Leader. State: Replica: c6bb3638781e4fb6a225791e7d23d00e, State: Running, Role: LEADER
I20260808 14:29:22.199271 16325 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bb3638781e4fb6a225791e7d23d00e" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36447 } }
I20260808 14:29:22.199795 16320 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:29:22.201090 16326 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6bb3638781e4fb6a225791e7d23d00e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bb3638781e4fb6a225791e7d23d00e" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36447 } } }
I20260808 14:29:22.201117 16327 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6bb3638781e4fb6a225791e7d23d00e. Latest consensus state: current_term: 1 leader_uuid: "c6bb3638781e4fb6a225791e7d23d00e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bb3638781e4fb6a225791e7d23d00e" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36447 } } }
I20260808 14:29:22.201215 16326 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:22.201228 16327 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:22.201634 16336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:29:22.204921 16336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:29:22.211046 16336 catalog_manager.cc:1347] Generated new cluster ID: 1252ecc986294bbea3950873816614be
I20260808 14:29:22.211128 16336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:29:22.237094 16336 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:29:22.238620 16336 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:29:22.247036 16336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e: Generated new TSK 0
I20260808 14:29:22.247871 16336 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.125.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
I20260808 14:29:22.281786 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:36447
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:22.398468 16349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:22.398840 16349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:22.399008 16349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:22.404688 16349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:22.404767 16349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:22.404893 16349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:29:22.411854 16349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:36447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:22.413333 16349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:22.414812 16349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:22.427997 16357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:22.428278 16354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:22.428020 16355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:22.428575 16349 server_base.cc:1034] running on GCE node
I20260808 14:29:22.429139 16349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:22.430059 16349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:22.431277 16349 hybrid_clock.cc:648] HybridClock initialized: now 1786199362431256 us; error 53 us; skew 500 ppm
Aug 08 14:29:22 dist-test-slave-w206 krb5kdc[16236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199362, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:22.435143 16349 init.cc:377] Logged in from keytab as kudu/127.7.125.65@KRBTEST.COM (short username kudu)
I20260808 14:29:22.436331 16349 webserver.cc:460] Webserver started at http://127.7.125.65:42157/ using document root <none> and password file <none>
I20260808 14:29:22.437141 16349 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:22.437261 16349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:22.437575 16349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:22.440529 16349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "ee3eb712d17144d883f39a1bc583f066"
format_stamp: "Formatted at 2026-08-08 14:29:22 on dist-test-slave-w206"
server_key: "396bcaa9d853f4d213b6ca988c00dc3f"
server_key_iv: "7bebd583fdbcfd2e8fa60663dc48b1b8"
server_key_version: "encryptionkey@0"
I20260808 14:29:22.441439 16349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "ee3eb712d17144d883f39a1bc583f066"
format_stamp: "Formatted at 2026-08-08 14:29:22 on dist-test-slave-w206"
server_key: "396bcaa9d853f4d213b6ca988c00dc3f"
server_key_iv: "7bebd583fdbcfd2e8fa60663dc48b1b8"
server_key_version: "encryptionkey@0"
I20260808 14:29:22.447388 16349 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260808 14:29:22.452152 16365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:22.454805 16349 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:29:22.455085 16349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "ee3eb712d17144d883f39a1bc583f066"
format_stamp: "Formatted at 2026-08-08 14:29:22 on dist-test-slave-w206"
server_key: "396bcaa9d853f4d213b6ca988c00dc3f"
server_key_iv: "7bebd583fdbcfd2e8fa60663dc48b1b8"
server_key_version: "encryptionkey@0"
I20260808 14:29:22.455328 16349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:22.509971 16349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:22.515189 16349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:22.515383 16349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:22.516119 16349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:22.517182 16349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:22.517277 16349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:22.517367 16349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:22.517403 16349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:22.531620 16349 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:42277
I20260808 14:29:22.531685 16479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:42277 every 8 connection(s)
I20260808 14:29:22.533387 16349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:29:22.540671 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 16349
I20260808 14:29:22.540941 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:29:22.541443 7669 external_mini_cluster.cc:1428] Setting key 1341e083f279def8399ce0b2a62af615
Aug 08 14:29:22 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199362, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:29:22.557166 16323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:22.536497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:53353 (local address 127.7.125.126:36447)
0808 14:29:22.536734 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:22.536747 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:22.538391 (+ 1644us) server_negotiation.cc:407] Connection header received
0808 14:29:22.539723 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:22.539726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:22.539807 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:22.539912 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:22.542036 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:22.542869 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:22.545115 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:22.545832 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:22.546067 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:22.550486 (+ 4419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:22.550497 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:22.550499 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:22.550538 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:22.553210 (+ 2672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:22.554258 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:22.554269 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:22.554270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:22.554415 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:22.555123 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:22.555130 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:22.555134 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:22.555472 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:22.555706 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:22.556640 (+ 934us) server_negotiation.cc:299] Negotiation successful
0808 14:29:22.556888 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
WARNING: no policy specified for kudu/127.7.125.66@KRBTEST.COM; defaulting to no policy
I20260808 14:29:22.558789 16482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:22.537165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.65:53353)
0808 14:29:22.538134 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:22.538201 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:22.539467 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:22.540133 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:22.540142 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:22.540766 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:22.541753 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:22.541781 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:22.543154 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:22.543158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:22.544920 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:22.544930 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:22.546135 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:22.546139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:22.546391 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:22.547501 (+ 1110us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:22.547554 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:22.550185 (+ 2631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:22.553527 (+ 3342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:22.553535 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:22.553569 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:22.554028 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:22.554600 (+ 572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:22.554603 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:22.554605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:22.554861 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:22.555707 (+ 846us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:22.555728 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:22.556188 (+ 460us) client_negotiation.cc:769] Sending connection context
0808 14:29:22.556538 (+ 350us) client_negotiation.cc:240] Negotiation successful
0808 14:29:22.557179 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":179,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.66@KRBTEST.COM" created.
I20260808 14:29:22.561283 16480 heartbeater.cc:347] Connected to a master server at 127.7.125.126:36447
I20260808 14:29:22.561760 16480 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:22.562659 16480 heartbeater.cc:511] Master 127.7.125.126:36447 requested a full tablet report, sending...
I20260808 14:29:22.565305 16284 ts_manager.cc:194] Registered new tserver with Master: ee3eb712d17144d883f39a1bc583f066 (127.7.125.65:42277)
I20260808 14:29:22.567229 16284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:53353
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
I20260808 14:29:22.587092 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:36447
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:22.805212 16486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:22.805966 16486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:22.806118 16486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:22.812950 16486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:22.813141 16486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:22.813340 16486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:29:22.826267 16486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:36447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:22.829365 16486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:22.832569 16486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:22.855244 16492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:22.855257 16495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:22.855244 16493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:22.856021 16486 server_base.cc:1034] running on GCE node
I20260808 14:29:22.857010 16486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:22.858217 16486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:22.859495 16486 hybrid_clock.cc:648] HybridClock initialized: now 1786199362859453 us; error 70 us; skew 500 ppm
Aug 08 14:29:22 dist-test-slave-w206 krb5kdc[16236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199362, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:22.864861 16486 init.cc:377] Logged in from keytab as kudu/127.7.125.66@KRBTEST.COM (short username kudu)
I20260808 14:29:22.866411 16486 webserver.cc:460] Webserver started at http://127.7.125.66:36401/ using document root <none> and password file <none>
I20260808 14:29:22.867501 16486 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:22.867635 16486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:22.868121 16486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:22.872403 16486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "38c263bb8ab44b15a34b144eefedbec2"
format_stamp: "Formatted at 2026-08-08 14:29:22 on dist-test-slave-w206"
server_key: "2c8f0391332862c6e9c40202c372bab4"
server_key_iv: "1fcda3ae2ac6856594d172907299bf4d"
server_key_version: "encryptionkey@0"
I20260808 14:29:22.873709 16486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "38c263bb8ab44b15a34b144eefedbec2"
format_stamp: "Formatted at 2026-08-08 14:29:22 on dist-test-slave-w206"
server_key: "2c8f0391332862c6e9c40202c372bab4"
server_key_iv: "1fcda3ae2ac6856594d172907299bf4d"
server_key_version: "encryptionkey@0"
I20260808 14:29:22.881760 16486 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260808 14:29:22.886454 16502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:22.888764 16486 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:29:22.889024 16486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "38c263bb8ab44b15a34b144eefedbec2"
format_stamp: "Formatted at 2026-08-08 14:29:22 on dist-test-slave-w206"
server_key: "2c8f0391332862c6e9c40202c372bab4"
server_key_iv: "1fcda3ae2ac6856594d172907299bf4d"
server_key_version: "encryptionkey@0"
I20260808 14:29:22.889204 16486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:22.905084 16486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:22.909902 16486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:22.910249 16486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:22.911247 16486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:22.912582 16486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:22.912658 16486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:22.912765 16486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:22.912838 16486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:22.929862 16486 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:32777
I20260808 14:29:22.929900 16621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:32777 every 8 connection(s)
I20260808 14:29:22.931566 16486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:29:22.940313 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 16486
I20260808 14:29:22.940572 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:29:22.941002 7669 external_mini_cluster.cc:1428] Setting key 06a529bb190248ecc3ee2828e958909e
Aug 08 14:29:22 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199362, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:29:22.956187 16323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:22.934554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:49529 (local address 127.7.125.126:36447)
0808 14:29:22.934741 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:22.934744 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:22.936472 (+ 1728us) server_negotiation.cc:407] Connection header received
0808 14:29:22.937792 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:22.937794 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:22.937872 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:22.938021 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:22.940255 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:22.941183 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:22.943322 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:22.943685 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:22.943806 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:22.948694 (+ 4888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:22.948712 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:22.948719 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:22.948779 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:22.952186 (+ 3407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:22.953082 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:22.953100 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:22.953105 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:22.953317 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:22.953904 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:22.953910 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:22.953913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:22.954442 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:22.954669 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:22.955634 (+ 965us) server_negotiation.cc:299] Negotiation successful
0808 14:29:22.955887 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 14:29:22.957340 16624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:22.935240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.66:49529)
0808 14:29:22.936234 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:22.936298 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:22.937499 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:22.938287 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:22.938296 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:22.938903 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:22.940009 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:22.940045 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:22.941430 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:22.941434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:22.943132 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:22.943149 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:22.943896 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:22.943901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:22.944293 (+ 392us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:22.945316 (+ 1023us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:22.945368 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:22.948376 (+ 3008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:22.952397 (+ 4021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:22.952415 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:22.952451 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:22.952914 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:22.953483 (+ 569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:22.953485 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:22.953494 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:22.953730 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:22.954645 (+ 915us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:22.954650 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:22.955160 (+ 510us) client_negotiation.cc:769] Sending connection context
0808 14:29:22.955444 (+ 284us) client_negotiation.cc:240] Negotiation successful
0808 14:29:22.955889 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"thread_start_us":161,"threads_started":1}
WARNING: no policy specified for kudu/127.7.125.67@KRBTEST.COM; defaulting to no policy
I20260808 14:29:22.959323 16622 heartbeater.cc:347] Connected to a master server at 127.7.125.126:36447
I20260808 14:29:22.959728 16622 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:22.960660 16622 heartbeater.cc:511] Master 127.7.125.126:36447 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.67@KRBTEST.COM" created.
I20260808 14:29:22.962422 16284 ts_manager.cc:194] Registered new tserver with Master: 38c263bb8ab44b15a34b144eefedbec2 (127.7.125.66:32777)
I20260808 14:29:22.963637 16284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:49529
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
I20260808 14:29:22.988660 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:36447
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:23.186607 16627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:23.187146 16627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:23.187280 16627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:23.193099 16627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:23.193203 16627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:23.193332 16627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:29:23.201691 16627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:36447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:23.203687 16627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:23.205458 16627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:23.221894 16632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:23.221916 16635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:23.222185 16627 server_base.cc:1034] running on GCE node
W20260808 14:29:23.222025 16633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:23.223387 16627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:23.224602 16627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:23.225957 16627 hybrid_clock.cc:648] HybridClock initialized: now 1786199363225911 us; error 100 us; skew 500 ppm
Aug 08 14:29:23 dist-test-slave-w206 krb5kdc[16236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199363, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:23.231655 16627 init.cc:377] Logged in from keytab as kudu/127.7.125.67@KRBTEST.COM (short username kudu)
I20260808 14:29:23.233368 16627 webserver.cc:460] Webserver started at http://127.7.125.67:43195/ using document root <none> and password file <none>
I20260808 14:29:23.234422 16627 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:23.234643 16627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:23.235134 16627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:23.239401 16627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "ddd44ff9cf39418a972750801173506e"
format_stamp: "Formatted at 2026-08-08 14:29:23 on dist-test-slave-w206"
server_key: "26c48343e690d95b79a61c3e237b629f"
server_key_iv: "348d5afe0feadb52654cf3667155e790"
server_key_version: "encryptionkey@0"
I20260808 14:29:23.240572 16627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "ddd44ff9cf39418a972750801173506e"
format_stamp: "Formatted at 2026-08-08 14:29:23 on dist-test-slave-w206"
server_key: "26c48343e690d95b79a61c3e237b629f"
server_key_iv: "348d5afe0feadb52654cf3667155e790"
server_key_version: "encryptionkey@0"
I20260808 14:29:23.248775 16627 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.001s sys 0.004s
I20260808 14:29:23.254114 16643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:23.256613 16627 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260808 14:29:23.256858 16627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "ddd44ff9cf39418a972750801173506e"
format_stamp: "Formatted at 2026-08-08 14:29:23 on dist-test-slave-w206"
server_key: "26c48343e690d95b79a61c3e237b629f"
server_key_iv: "348d5afe0feadb52654cf3667155e790"
server_key_version: "encryptionkey@0"
I20260808 14:29:23.257114 16627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:23.278074 16627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:23.283702 16627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:23.284169 16627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:23.285354 16627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:23.286962 16627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:23.287127 16627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260808 14:29:23.287190 16627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:23.287253 16627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:23.306217 16627 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:37401
I20260808 14:29:23.306254 16762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:37401 every 8 connection(s)
I20260808 14:29:23.308015 16627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:29:23.312662 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 16627
I20260808 14:29:23.312912 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:29:23.313511 7669 external_mini_cluster.cc:1428] Setting key 0ceea969ccbaf371538c3614095148b5
Aug 08 14:29:23 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199363, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:29:23.332463 16323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:23.311044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:42119 (local address 127.7.125.126:36447)
0808 14:29:23.311273 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:23.311280 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:23.313527 (+ 2247us) server_negotiation.cc:407] Connection header received
0808 14:29:23.315145 (+ 1618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:23.315159 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:23.315258 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:23.315459 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:23.317544 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:23.318366 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:23.320343 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:23.320695 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:23.320890 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:23.325233 (+ 4343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:23.325246 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:23.325254 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:23.325308 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:23.328651 (+ 3343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:23.329621 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:23.329624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:23.329625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:23.329810 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:23.330541 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:23.330545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:23.330546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:23.330913 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:23.331195 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:23.332058 (+ 863us) server_negotiation.cc:299] Negotiation successful
0808 14:29:23.332234 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 14:29:23.333952 16765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:23.311929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.67:42119)
0808 14:29:23.313233 (+ 1304us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:23.313333 (+ 100us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:23.314722 (+ 1389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:23.315760 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:23.315769 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:23.316379 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:23.317313 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:23.317340 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:23.318594 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:23.318598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:23.320090 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:23.320100 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:23.320990 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:23.320995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:23.321276 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:23.322230 (+ 954us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:23.322267 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:23.324947 (+ 2680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:23.328839 (+ 3892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:23.328850 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:23.328879 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:23.329435 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:23.330004 (+ 569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:23.330010 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:23.330025 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:23.330345 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:23.331170 (+ 825us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:23.331176 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:23.331652 (+ 476us) client_negotiation.cc:769] Sending connection context
0808 14:29:23.331938 (+ 286us) client_negotiation.cc:240] Negotiation successful
0808 14:29:23.332412 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"thread_start_us":193,"threads_started":1}
I20260808 14:29:23.335947 16763 heartbeater.cc:347] Connected to a master server at 127.7.125.126:36447
I20260808 14:29:23.336390 16763 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:23.337376 16763 heartbeater.cc:511] Master 127.7.125.126:36447 requested a full tablet report, sending...
I20260808 14:29:23.339212 16284 ts_manager.cc:194] Registered new tserver with Master: ddd44ff9cf39418a972750801173506e (127.7.125.67:37401)
I20260808 14:29:23.340181 16284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:42119
I20260808 14:29:23.342881 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.125.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.125.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.125.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.65.keytab.
Entry for principal oryx/127.7.125.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.65.keytab.
WARNING: no policy specified for oryx/127.7.125.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.125.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.125.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.66.keytab.
Entry for principal oryx/127.7.125.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.66.keytab.
WARNING: no policy specified for oryx/127.7.125.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.125.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.125.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.67.keytab.
Entry for principal oryx/127.7.125.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.67.keytab.
I20260808 14:29:23.453737 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 16349
I20260808 14:29:23.461126 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 16486
I20260808 14:29:23.467875 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 16627
I20260808 14:29:23.474889 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 16252
I20260808 14:29:23.483425 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:36447
--webserver_interface=127.7.125.126
--webserver_port=42873
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:36447
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:23.597924 16772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:23.598219 16772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:23.598299 16772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:23.602497 16772 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:29:23.602566 16772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:23.602587 16772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:23.602604 16772 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:29:23.602646 16772 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:29:23.608142 16772 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:36447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:36447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=42873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:23.609208 16772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:23.610253 16772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:23.619040 16777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:23.619086 16778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:23.619246 16772 server_base.cc:1034] running on GCE node
W20260808 14:29:23.619086 16780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:23.619897 16772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:23.621012 16772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:23.622198 16772 hybrid_clock.cc:648] HybridClock initialized: now 1786199363622185 us; error 25 us; skew 500 ppm
Aug 08 14:29:23 dist-test-slave-w206 krb5kdc[16236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199363, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:23.625089 16772 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:29:23.626083 16772 webserver.cc:460] Webserver started at http://127.7.125.126:42873/ using document root <none> and password file <none>
I20260808 14:29:23.626659 16772 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:23.626775 16772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:23.630780 16772 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:29:23.633061 16787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:23.634438 16772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:29:23.634588 16772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "c6bb3638781e4fb6a225791e7d23d00e"
format_stamp: "Formatted at 2026-08-08 14:29:22 on dist-test-slave-w206"
server_key: "6a7e26e66be96bfdc07a32b903892a67"
server_key_iv: "65e07bd13f0be13d2587aa5d3d979d4b"
server_key_version: "encryptionkey@0"
I20260808 14:29:23.635185 16772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:23.651117 16772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:23.654078 16772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:23.654287 16772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:23.661636 16772 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:36447
I20260808 14:29:23.661661 16841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:36447 every 8 connection(s)
I20260808 14:29:23.662694 16772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:29:23.666812 16842 sys_catalog.cc:263] Verifying existing consensus state
I20260808 14:29:23.668454 16842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e: Bootstrap starting.
I20260808 14:29:23.670784 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 16772
I20260808 14:29:23.671283 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:42277
--local_ip_for_outbound_sockets=127.7.125.65
--tserver_master_addrs=127.7.125.126:36447
--webserver_port=42157
--webserver_interface=127.7.125.65
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.125.65
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:29:23.686113 16842 log.cc:824] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:23.692795 16842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:29:23.693192 16842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e: Bootstrap complete.
I20260808 14:29:23.697690 16842 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bb3638781e4fb6a225791e7d23d00e" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36447 } }
I20260808 14:29:23.698124 16842 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6bb3638781e4fb6a225791e7d23d00e, State: Initialized, Role: FOLLOWER
I20260808 14:29:23.698611 16842 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bb3638781e4fb6a225791e7d23d00e" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36447 } }
I20260808 14:29:23.698753 16842 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:29:23.698843 16842 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:29:23.699009 16842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [term 1 FOLLOWER]: Advancing to term 2
I20260808 14:29:23.700990 16842 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bb3638781e4fb6a225791e7d23d00e" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36447 } }
I20260808 14:29:23.701321 16842 leader_election.cc:302] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6bb3638781e4fb6a225791e7d23d00e; no voters:
I20260808 14:29:23.701638 16842 leader_election.cc:288] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 14:29:23.701771 16846 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [term 2 FOLLOWER]: Leader election won for term 2
I20260808 14:29:23.702026 16846 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [term 2 LEADER]: Becoming Leader. State: Replica: c6bb3638781e4fb6a225791e7d23d00e, State: Running, Role: LEADER
I20260808 14:29:23.702420 16846 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bb3638781e4fb6a225791e7d23d00e" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36447 } }
I20260808 14:29:23.702876 16842 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:29:23.703480 16846 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6bb3638781e4fb6a225791e7d23d00e. Latest consensus state: current_term: 2 leader_uuid: "c6bb3638781e4fb6a225791e7d23d00e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bb3638781e4fb6a225791e7d23d00e" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36447 } } }
I20260808 14:29:23.703598 16846 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:23.704202 16847 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c6bb3638781e4fb6a225791e7d23d00e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bb3638781e4fb6a225791e7d23d00e" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36447 } } }
I20260808 14:29:23.704319 16847 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:23.705044 16852 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:29:23.708354 16852 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:29:23.708994 16852 catalog_manager.cc:1259] Loaded cluster ID: 1252ecc986294bbea3950873816614be
I20260808 14:29:23.709055 16852 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:29:23.711388 16852 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:29:23.712040 16852 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c6bb3638781e4fb6a225791e7d23d00e: Loaded TSK: 0
I20260808 14:29:23.713233 16852 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 14:29:23.789333 16844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:23.789574 16844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:23.789618 16844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:23.793758 16844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:23.793824 16844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:23.793882 16844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:29:23.799314 16844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.65.keytab
--principal=oryx/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:42277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=42157
--tserver_master_addrs=127.7.125.126:36447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:23.800359 16844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:23.801369 16844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:23.811252 16870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:23.811285 16869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:23.811308 16844 server_base.cc:1034] running on GCE node
W20260808 14:29:23.811259 16872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:23.811890 16844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:23.812561 16844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:23.813759 16844 hybrid_clock.cc:648] HybridClock initialized: now 1786199363813732 us; error 45 us; skew 500 ppm
Aug 08 14:29:23 dist-test-slave-w206 krb5kdc[16236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199363, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:23.816764 16844 init.cc:377] Logged in from keytab as oryx/127.7.125.65@KRBTEST.COM (short username oryx)
I20260808 14:29:23.817713 16844 webserver.cc:460] Webserver started at http://127.7.125.65:42157/ using document root <none> and password file <none>
I20260808 14:29:23.818277 16844 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:23.818353 16844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:23.822309 16844 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:29:23.824409 16879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:23.825682 16844 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:29:23.825773 16844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "ee3eb712d17144d883f39a1bc583f066"
format_stamp: "Formatted at 2026-08-08 14:29:22 on dist-test-slave-w206"
server_key: "396bcaa9d853f4d213b6ca988c00dc3f"
server_key_iv: "7bebd583fdbcfd2e8fa60663dc48b1b8"
server_key_version: "encryptionkey@0"
I20260808 14:29:23.826337 16844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:23.848671 16844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:23.851744 16844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:23.851950 16844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:23.852596 16844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:23.853447 16844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:23.853502 16844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:23.853570 16844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:23.853600 16844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:23.862879 16844 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:42277
I20260808 14:29:23.862916 16998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:42277 every 8 connection(s)
I20260808 14:29:23.863916 16844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:29:23.868788 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 16844
I20260808 14:29:23.869350 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:32777
--local_ip_for_outbound_sockets=127.7.125.66
--tserver_master_addrs=127.7.125.126:36447
--webserver_port=36401
--webserver_interface=127.7.125.66
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.125.66
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 14:29:23 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:23 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:23.877588 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:23.865982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.65:56637)
0808 14:29:23.866711 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:23.866766 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:23.867603 (+ 837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:23.868684 (+ 1081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:23.868696 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:23.869201 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:23.869888 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:23.869912 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:23.871373 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:23.871376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:23.872218 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:23.872228 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:23.873311 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:23.873313 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:23.873460 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:23.874098 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:23.874115 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:23.876817 (+ 2702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":146,"threads_started":1}
W20260808 14:29:23.878078 16999 heartbeater.cc:644] Failed to heartbeat to 127.7.125.126:36447 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.125.126:36447: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
W20260808 14:29:23.878149 17001 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:23.865911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:56637 (local address 127.7.125.126:36447)
0808 14:29:23.866350 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:23.866375 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:23.866883 (+ 508us) server_negotiation.cc:407] Connection header received
0808 14:29:23.867971 (+ 1088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:23.867982 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:23.868262 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:23.868556 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:23.870085 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:23.871231 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:23.872356 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:23.873140 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:23.873330 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:23.877161 (+ 3831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.65:56637: BlockingRecv error: recv got EOF from 127.7.125.65:56637 (error 108)
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":133,"threads_started":1}
Aug 08 14:29:23 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:23 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:23.884855 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:23.878634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.65:35063)
0808 14:29:23.878820 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:23.878834 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:23.878924 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:23.879296 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:23.879298 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:23.879434 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:23.879523 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:23.879527 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:23.880653 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:23.880655 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:23.881185 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:23.881192 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:23.881827 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:23.881830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:23.881907 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:23.882404 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:23.882415 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:23.884751 (+ 2336us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101}
W20260808 14:29:23.885035 17001 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:23.878851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:35063 (local address 127.7.125.126:36447)
0808 14:29:23.879027 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:23.879034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:23.879053 (+ 19us) server_negotiation.cc:407] Connection header received
0808 14:29:23.879102 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:23.879106 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:23.879166 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:23.879262 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:23.879636 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:23.880538 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:23.881303 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:23.881712 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:23.881833 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:23.884922 (+ 3089us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.65:35063: BlockingRecv error: recv got EOF from 127.7.125.65:35063 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 14:29:23 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:23 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:23.890954 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:23.885382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.65:35341)
0808 14:29:23.885705 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:23.885716 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:23.885786 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:23.886030 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:23.886032 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:23.886156 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:23.886227 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:23.886231 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:23.887337 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:23.887340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:23.887848 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:23.887855 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:23.888402 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:23.888404 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:23.888466 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:23.888982 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:23.888993 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:23.890842 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":101,"threads_started":1}
W20260808 14:29:23.891158 17001 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:23.885509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:35341 (local address 127.7.125.126:36447)
0808 14:29:23.885629 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:23.885634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:23.885771 (+ 137us) server_negotiation.cc:407] Connection header received
0808 14:29:23.885867 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:23.885872 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:23.885928 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:23.886003 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:23.886388 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:23.887207 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:23.887956 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:23.888296 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:23.888387 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:23.891050 (+ 2663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.65:35341: BlockingRecv error: recv got EOF from 127.7.125.65:35341 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 14:29:23.891280 16999 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 14:29:23.987159 17003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:23.987435 17003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:23.987480 17003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:23.991482 17003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:23.991539 17003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:23.991595 17003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:29:23.996906 17003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.66.keytab
--principal=oryx/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:32777
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=36401
--tserver_master_addrs=127.7.125.126:36447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:23.998030 17003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:23.999087 17003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:24.008972 17009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:24.008980 17010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:24.008980 17012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:24.009362 17003 server_base.cc:1034] running on GCE node
I20260808 14:29:24.009749 17003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:24.010416 17003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:24.011600 17003 hybrid_clock.cc:648] HybridClock initialized: now 1786199364011588 us; error 24 us; skew 500 ppm
Aug 08 14:29:24 dist-test-slave-w206 krb5kdc[16236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199364, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:24.014410 17003 init.cc:377] Logged in from keytab as oryx/127.7.125.66@KRBTEST.COM (short username oryx)
I20260808 14:29:24.015336 17003 webserver.cc:460] Webserver started at http://127.7.125.66:36401/ using document root <none> and password file <none>
I20260808 14:29:24.015946 17003 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:24.016048 17003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:24.019939 17003 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:29:24.021940 17019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:24.023196 17003 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:29:24.023326 17003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "38c263bb8ab44b15a34b144eefedbec2"
format_stamp: "Formatted at 2026-08-08 14:29:22 on dist-test-slave-w206"
server_key: "2c8f0391332862c6e9c40202c372bab4"
server_key_iv: "1fcda3ae2ac6856594d172907299bf4d"
server_key_version: "encryptionkey@0"
I20260808 14:29:24.023886 17003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:24.045770 17003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:24.049335 17003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:24.049554 17003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:24.050201 17003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:24.051076 17003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:24.051124 17003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:24.051201 17003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:24.051239 17003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:24.060211 17003 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:32777
I20260808 14:29:24.060238 17136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:32777 every 8 connection(s)
I20260808 14:29:24.061213 17003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:29:24.067682 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 17003
I20260808 14:29:24.068154 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:37401
--local_ip_for_outbound_sockets=127.7.125.67
--tserver_master_addrs=127.7.125.126:36447
--webserver_port=43195
--webserver_interface=127.7.125.67
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.125.67
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 14:29:24 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:24 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 14:29:24.073551 17001 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:24.063215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:41861 (local address 127.7.125.126:36447)
0808 14:29:24.063365 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:24.063369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:24.064648 (+ 1279us) server_negotiation.cc:407] Connection header received
0808 14:29:24.065570 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:24.065574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:24.065639 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:24.065755 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:24.066996 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.067536 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.068871 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.069104 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.069223 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:24.073372 (+ 4149us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.66:41861: BlockingRecv error: recv got EOF from 127.7.125.66:41861 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 14:29:24.074098 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:24.063614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.66:41861)
0808 14:29:24.064493 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:24.064538 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:24.065338 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:24.065947 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:24.065955 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:24.066325 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:24.066814 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:24.066828 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.067673 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.067682 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:24.068737 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:24.068747 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.069267 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.069271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:24.069456 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:24.070137 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:24.070161 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:24.073146 (+ 2985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":528,"threads_started":1}
W20260808 14:29:24.074856 17137 heartbeater.cc:644] Failed to heartbeat to 127.7.125.126:36447 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.125.126:36447: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Aug 08 14:29:24 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:24 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:24.081099 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:24.075327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.66:40575)
0808 14:29:24.075587 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:24.075600 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:24.075670 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:24.076008 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:24.076010 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:24.076151 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:24.076242 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:24.076246 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.077365 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.077367 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:24.077892 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:24.077899 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.078540 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.078542 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:24.078608 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:24.079140 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:24.079151 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:24.080997 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195}
W20260808 14:29:24.081283 17001 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:24.075462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:40575 (local address 127.7.125.126:36447)
0808 14:29:24.075624 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:24.075631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:24.075650 (+ 19us) server_negotiation.cc:407] Connection header received
0808 14:29:24.075791 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:24.075794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:24.075874 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:24.075976 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:24.076358 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.077244 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.078011 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.078422 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.078556 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:24.081164 (+ 2608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.66:40575: BlockingRecv error: recv got EOF from 127.7.125.66:40575 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 14:29:24 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:24 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:24.086966 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:24.081684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.66:43127)
0808 14:29:24.081907 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:24.081921 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:24.082026 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:24.082277 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:24.082279 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:24.082451 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:24.082520 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:24.082525 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.083361 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.083366 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:24.083892 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:24.083899 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.084348 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.084351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:24.084435 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:24.084942 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:24.084953 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:24.086814 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151}
W20260808 14:29:24.087235 17001 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:24.081760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:43127 (local address 127.7.125.126:36447)
0808 14:29:24.081874 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:24.081879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:24.081987 (+ 108us) server_negotiation.cc:407] Connection header received
0808 14:29:24.082127 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:24.082130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:24.082176 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:24.082260 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:24.082631 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.083248 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.084018 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.084230 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.084341 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:24.086939 (+ 2598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.66:43127: BlockingRecv error: recv got EOF from 127.7.125.66:43127 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 14:29:24.087301 17137 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 14:29:24.187772 17140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:24.188105 17140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:24.188184 17140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:24.192267 17140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:24.192358 17140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:24.192456 17140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:29:24.197743 17140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/krb5kdc/oryx_127.7.125.67.keytab
--principal=oryx/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:37401
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=43195
--tserver_master_addrs=127.7.125.126:36447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:24.199054 17140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:24.200135 17140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:24.210216 17146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:24.210263 17140 server_base.cc:1034] running on GCE node
W20260808 14:29:24.210210 17148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:24.210213 17145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:24.210916 17140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:24.211588 17140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:24.212750 17140 hybrid_clock.cc:648] HybridClock initialized: now 1786199364212730 us; error 32 us; skew 500 ppm
Aug 08 14:29:24 dist-test-slave-w206 krb5kdc[16236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199364, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:24.215785 17140 init.cc:377] Logged in from keytab as oryx/127.7.125.67@KRBTEST.COM (short username oryx)
I20260808 14:29:24.216758 17140 webserver.cc:460] Webserver started at http://127.7.125.67:43195/ using document root <none> and password file <none>
I20260808 14:29:24.217271 17140 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:24.217377 17140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:24.221269 17140 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:29:24.223485 17155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:24.224740 17140 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:29:24.224843 17140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "ddd44ff9cf39418a972750801173506e"
format_stamp: "Formatted at 2026-08-08 14:29:23 on dist-test-slave-w206"
server_key: "26c48343e690d95b79a61c3e237b629f"
server_key_iv: "348d5afe0feadb52654cf3667155e790"
server_key_version: "encryptionkey@0"
I20260808 14:29:24.225441 17140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:24.247824 17140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:24.250773 17140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:24.250923 17140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:24.251561 17140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:24.252373 17140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:24.252421 17140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:24.252465 17140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:24.252486 17140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:24.261732 17140 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:37401
I20260808 14:29:24.261757 17270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:37401 every 8 connection(s)
I20260808 14:29:24.262768 17140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:29:24.266202 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 17140
Aug 08 14:29:24 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:24 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 14:29:24.274933 17001 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:24.264655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:34931 (local address 127.7.125.126:36447)
0808 14:29:24.264836 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:24.264841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:24.265887 (+ 1046us) server_negotiation.cc:407] Connection header received
0808 14:29:24.266917 (+ 1030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:24.266921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:24.266997 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:24.267089 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:24.268734 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.269434 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.270934 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.271266 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.271379 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:24.274822 (+ 3443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.67:34931: BlockingRecv error: recv got EOF from 127.7.125.67:34931 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 14:29:24.275740 17273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:24.265076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.67:34931)
0808 14:29:24.265730 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:24.265772 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:24.266662 (+ 890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:24.267402 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:24.267412 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:24.267888 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:24.268607 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:24.268625 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.269552 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.269556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:24.270815 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:24.270824 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.271476 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.271482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:24.271636 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:24.272151 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:24.272178 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:24.274669 (+ 2491us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":119,"threads_started":1}
W20260808 14:29:24.276237 17271 heartbeater.cc:644] Failed to heartbeat to 127.7.125.126:36447 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.125.126:36447: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
I20260808 14:29:24.277529 17275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:24.267264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54348 (local address 127.7.125.126:36447)
0808 14:29:24.267505 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:24.267508 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:24.267519 (+ 11us) server_negotiation.cc:407] Connection header received
0808 14:29:24.267551 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:24.267553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:24.267586 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:24.267660 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:24.268573 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.269407 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.270719 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.271066 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.271172 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:24.272839 (+ 1667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:24.272845 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:24.272859 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:24.272884 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:24.274624 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:24.275254 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:24.275259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:24.275263 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:24.275383 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:24.275676 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:24.275678 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:24.275679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:24.276148 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:24.276339 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:24.276996 (+ 657us) server_negotiation.cc:299] Negotiation successful
0808 14:29:24.277334 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":177,"thread_start_us":116,"threads_started":1}
Aug 08 14:29:24 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:24 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:24.281821 17273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:24.276672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.67:48353)
0808 14:29:24.276847 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:24.276861 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:24.276945 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:24.277162 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:24.277166 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:24.277334 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:24.277442 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:24.277448 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.278222 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.278224 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:24.278752 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:24.278759 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.279290 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.279293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:24.279384 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:24.279836 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:24.279852 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:24.281736 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105}
W20260808 14:29:24.282016 17001 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:24.276767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:48353 (local address 127.7.125.126:36447)
0808 14:29:24.276956 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:24.276959 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:24.276991 (+ 32us) server_negotiation.cc:407] Connection header received
0808 14:29:24.277020 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:24.277022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:24.277056 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:24.277117 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:24.277554 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.278124 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.278870 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.279173 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.279262 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:24.281883 (+ 2621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.67:48353: BlockingRecv error: recv got EOF from 127.7.125.67:48353 (error 108)
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":116}
Aug 08 14:29:24 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:24 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:24.286675 17273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:24.282328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.67:42713)
0808 14:29:24.282445 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:24.282458 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:24.282528 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:24.282737 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:24.282739 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:24.282860 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:24.282926 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:24.282929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.283643 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.283646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:24.284147 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:24.284154 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.284549 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.284551 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:24.284605 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:24.284992 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:24.285002 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:24.286591 (+ 1589us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 14:29:24.286827 17001 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:24.282443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:42713 (local address 127.7.125.126:36447)
0808 14:29:24.282571 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:24.282574 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:24.282584 (+ 10us) server_negotiation.cc:407] Connection header received
0808 14:29:24.282612 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:24.282614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:24.282646 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:24.282710 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:24.283067 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.283557 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.284242 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.284459 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.284531 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:24.286713 (+ 2182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.67:42713: BlockingRecv error: recv got EOF from 127.7.125.67:42713 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 14:29:24.287032 17271 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 14:29:24 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:24 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:24.897364 17276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:24.891961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.65:39451)
0808 14:29:24.892238 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:24.892253 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:24.892367 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:24.892612 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:24.892614 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:24.892792 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:24.892907 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:24.892912 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.893648 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.893650 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:24.894236 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:24.894244 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.894685 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.894687 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:24.894759 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:24.895350 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:24.895362 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:24.897204 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":97,"threads_started":1}
W20260808 14:29:24.897500 17277 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:24.892021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:39451 (local address 127.7.125.126:36447)
0808 14:29:24.892269 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:24.892273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:24.892329 (+ 56us) server_negotiation.cc:407] Connection header received
0808 14:29:24.892449 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:24.892452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:24.892496 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:24.892577 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:24.893009 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.893549 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.894344 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:24.894585 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:24.894663 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:24.897352 (+ 2689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.65:39451: BlockingRecv error: recv got EOF from 127.7.125.65:39451 (error 108)
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":92,"threads_started":1}
Aug 08 14:29:25 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:25 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:25.093060 17278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:25.087913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.66:41085)
0808 14:29:25.088185 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:25.088198 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:25.088298 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:25.088571 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:25.088573 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:25.088750 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:25.088847 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:25.088852 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:25.089559 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:25.089561 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:25.090129 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:25.090155 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:25.090563 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:25.090565 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:25.090627 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:25.091145 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:25.091157 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:25.092926 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":101,"threads_started":1}
W20260808 14:29:25.093187 17277 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:25.087976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:41085 (local address 127.7.125.126:36447)
0808 14:29:25.088151 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:25.088156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:25.088271 (+ 115us) server_negotiation.cc:407] Connection header received
0808 14:29:25.088415 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:25.088418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:25.088461 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:25.088529 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:25.088944 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:25.089464 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:25.090249 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:25.090468 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:25.090542 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:25.093073 (+ 2531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.66:41085: BlockingRecv error: recv got EOF from 127.7.125.66:41085 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 08 14:29:25 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:25 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:25.292883 17279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:25.287600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.67:48929)
0808 14:29:25.287861 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:25.287881 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:25.287981 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:25.288222 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:25.288224 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:25.288408 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:25.288503 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:25.288508 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:25.289203 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:25.289205 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:25.289761 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:25.289769 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:25.290177 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:25.290180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:25.290242 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:25.290723 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:25.290734 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:25.292752 (+ 2018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":92,"threads_started":1}
W20260808 14:29:25.292973 17277 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:25.287682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:48929 (local address 127.7.125.126:36447)
0808 14:29:25.287818 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:25.287822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:25.287952 (+ 130us) server_negotiation.cc:407] Connection header received
0808 14:29:25.288070 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:25.288072 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:25.288117 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:25.288186 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:25.288605 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:25.289113 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:25.289867 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:25.290080 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:25.290157 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:25.292884 (+ 2727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.67:48929: BlockingRecv error: recv got EOF from 127.7.125.67:48929 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 14:29:25 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:25 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:25.909204 17280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:25.898257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.65:53799)
0808 14:29:25.898538 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:25.898551 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:25.898642 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:25.898989 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:25.898992 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:25.899178 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:25.899283 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:25.899289 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:25.900060 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:25.900063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:25.900623 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:25.900631 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:25.901047 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:25.901049 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:25.901111 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:25.907092 (+ 5981us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:25.907106 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:25.909029 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":99,"threads_started":1}
W20260808 14:29:25.909281 17281 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:25.898405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:53799 (local address 127.7.125.126:36447)
0808 14:29:25.898681 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:25.898684 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:25.898697 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:29:25.898750 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:25.898752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:25.898794 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:25.898878 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:25.899395 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:25.899955 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:25.900725 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:25.900950 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:25.901025 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:25.909157 (+ 8132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.65:53799: BlockingRecv error: recv got EOF from 127.7.125.65:53799 (error 108)
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":122,"threads_started":1}
Aug 08 14:29:26 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:26 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:26.100345 17282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:26.093926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.66:58727)
0808 14:29:26.094317 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:26.094330 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:26.094458 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:26.094738 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:26.094741 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:26.095021 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:26.095134 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:26.095140 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:26.095907 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:26.095911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:26.096468 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:26.096476 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:26.096939 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:26.096942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:26.097031 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:26.097658 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:26.097674 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:26.100112 (+ 2438us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":120,"threads_started":1}
W20260808 14:29:26.100449 17281 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:26.094016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:58727 (local address 127.7.125.126:36447)
0808 14:29:26.094193 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:26.094198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:26.094422 (+ 224us) server_negotiation.cc:407] Connection header received
0808 14:29:26.094569 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:26.094573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:26.094627 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:26.094716 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:26.095253 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:26.095786 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:26.096591 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:26.096820 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:26.096931 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:26.100304 (+ 3373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.66:58727: BlockingRecv error: recv got EOF from 127.7.125.66:58727 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 08 14:29:26 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:26 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:26.299117 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:26.293751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.67:56927)
0808 14:29:26.294030 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:26.294064 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:26.294192 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:26.294488 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:26.294491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:26.294687 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:26.294787 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:26.294792 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:26.295524 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:26.295527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:26.296082 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:26.296090 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:26.296485 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:26.296487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:26.296548 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:26.297032 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:26.297043 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:26.298914 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":95,"threads_started":1}
W20260808 14:29:26.299146 17281 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:26.293897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:56927 (local address 127.7.125.126:36447)
0808 14:29:26.294045 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:26.294050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:26.294151 (+ 101us) server_negotiation.cc:407] Connection header received
0808 14:29:26.294299 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:26.294304 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:26.294365 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:26.294449 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:26.294889 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:26.295426 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:26.296181 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:26.296389 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:26.296463 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:26.299054 (+ 2591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.67:56927: BlockingRecv error: recv got EOF from 127.7.125.67:56927 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 14:29:26 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:26 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:26.915778 17285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:26.910118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.65:50097)
0808 14:29:26.910382 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:26.910400 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:26.910512 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:26.910790 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:26.910793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:26.911078 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:26.911181 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:26.911188 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:26.911968 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:26.911974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:26.912539 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:26.912547 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:26.913021 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:26.913024 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:26.913109 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:26.913598 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:26.913611 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:26.915581 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":97,"threads_started":1}
W20260808 14:29:26.915863 17287 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:26.910173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:50097 (local address 127.7.125.126:36447)
0808 14:29:26.910413 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:26.910416 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:26.910478 (+ 62us) server_negotiation.cc:407] Connection header received
0808 14:29:26.910590 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:26.910592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:26.910655 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:26.910737 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:26.911320 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:26.911857 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:26.912663 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:26.912910 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:26.913017 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:26.915711 (+ 2694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.65:50097: BlockingRecv error: recv got EOF from 127.7.125.65:50097 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":89,"threads_started":1}
Aug 08 14:29:27 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:27 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:27.107841 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:27.101320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.66:58335)
0808 14:29:27.101647 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:27.101665 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:27.101787 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:27.102121 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:27.102126 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:27.102435 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:27.102562 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:27.102570 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:27.103787 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:27.103791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:27.104694 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:27.104705 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:27.105274 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:27.105277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:27.105338 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:27.105824 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:27.105837 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:27.107721 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":83,"threads_started":1}
W20260808 14:29:27.108107 17287 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:27.101417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:58335 (local address 127.7.125.126:36447)
0808 14:29:27.101616 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:27.101624 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:27.101756 (+ 132us) server_negotiation.cc:407] Connection header received
0808 14:29:27.101922 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:27.101925 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:27.101982 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:27.102060 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:27.102695 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:27.103616 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:27.104822 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:27.105159 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:27.105244 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:27.107856 (+ 2612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.66:58335: BlockingRecv error: recv got EOF from 127.7.125.66:58335 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 08 14:29:27 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:27 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:27.305826 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:27.299929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.67:58119)
0808 14:29:27.300288 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:27.300301 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:27.300442 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:27.300710 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:27.300713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:27.300929 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:27.301021 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:27.301027 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:27.301785 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:27.301790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:27.302349 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:27.302357 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:27.302875 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:27.302878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:27.302966 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:27.303661 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:27.303674 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:27.305653 (+ 1979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":115,"threads_started":1}
W20260808 14:29:27.305922 17287 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:27.300054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:58119 (local address 127.7.125.126:36447)
0808 14:29:27.300233 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:27.300238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:27.300379 (+ 141us) server_negotiation.cc:407] Connection header received
0808 14:29:27.300548 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:27.300550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:27.300605 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:27.300687 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:27.301140 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:27.301674 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:27.302502 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:27.302743 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:27.302869 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:27.305789 (+ 2920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.67:58119: BlockingRecv error: recv got EOF from 127.7.125.67:58119 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 08 14:29:27 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:27 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:27.922474 17290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:27.916590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.65:51787)
0808 14:29:27.916879 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:27.916897 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:27.917008 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:27.917247 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:27.917249 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:27.917436 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:27.917530 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:27.917535 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:27.918315 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:27.918319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:27.919192 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:27.919203 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:27.919677 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:27.919680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:27.919766 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:27.920341 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:27.920360 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:27.922289 (+ 1929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":70,"threads_started":1}
W20260808 14:29:27.922550 17291 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:27.916730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:51787 (local address 127.7.125.126:36447)
0808 14:29:27.917012 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:27.917016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:27.917029 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:29:27.917069 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:27.917071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:27.917131 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:27.917210 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:27.917630 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:27.918191 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:27.919318 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:27.919556 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:27.919671 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:27.922425 (+ 2754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.65:51787: BlockingRecv error: recv got EOF from 127.7.125.65:51787 (error 108)
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":100,"threads_started":1}
Aug 08 14:29:28 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:28 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:28.113674 17292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:28.108576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.66:43015)
0808 14:29:28.108854 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:28.108867 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:28.108968 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:28.109180 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:28.109182 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:28.109357 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:28.109452 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:28.109457 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:28.110160 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:28.110163 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:28.110703 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:28.110710 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:28.111168 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:28.111171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:28.111231 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:28.111687 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:28.111698 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:28.113559 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":71,"threads_started":1}
W20260808 14:29:28.113826 17291 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:28.108671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:43015 (local address 127.7.125.126:36447)
0808 14:29:28.108858 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:28.108862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:28.108920 (+ 58us) server_negotiation.cc:407] Connection header received
0808 14:29:28.109050 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:28.109052 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:28.109098 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:28.109161 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:28.109546 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:28.110067 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:28.110800 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:28.111073 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:28.111148 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:28.113699 (+ 2551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.66:43015: BlockingRecv error: recv got EOF from 127.7.125.66:43015 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 08 14:29:28 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:28 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:28.312566 17293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:28.306740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.67:33833)
0808 14:29:28.307082 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:28.307101 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:28.307225 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:28.307547 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:28.307550 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:28.307809 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:28.307930 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:28.307937 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:28.308854 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:28.308857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:28.309474 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:28.309486 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:28.309928 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:28.309931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:28.310008 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:28.310500 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:28.310511 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:28.312440 (+ 1929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":140,"threads_started":1}
W20260808 14:29:28.312825 17291 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:28.306948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:33833 (local address 127.7.125.126:36447)
0808 14:29:28.307141 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:28.307150 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:28.307172 (+ 22us) server_negotiation.cc:407] Connection header received
0808 14:29:28.307359 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:28.307363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:28.307423 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:28.307517 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:28.308068 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:28.308736 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:28.309591 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:28.309807 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:28.309884 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:28.312589 (+ 2705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.67:33833: BlockingRecv error: recv got EOF from 127.7.125.67:33833 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 08 14:29:28 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:28 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:28.928737 17294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:28.923357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.65:60735)
0808 14:29:28.923630 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:28.923644 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:28.923726 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:28.923976 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:28.923978 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:28.924167 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:28.924262 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:28.924268 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:28.925010 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:28.925012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:28.925567 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:28.925575 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:28.925981 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:28.925984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:28.926066 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:28.926586 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:28.926598 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:28.928620 (+ 2022us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":92,"threads_started":1}
W20260808 14:29:28.929015 17295 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:28.923452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:60735 (local address 127.7.125.126:36447)
0808 14:29:28.923714 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:28.923717 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:28.923731 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:29:28.923801 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:28.923803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:28.923843 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:28.923939 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:28.924374 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:28.924915 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:28.925663 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:28.925873 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:28.925960 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:28.928863 (+ 2903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.65:60735: BlockingRecv error: recv got EOF from 127.7.125.65:60735 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":110,"threads_started":1}
Aug 08 14:29:29 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:29 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:29.120095 17296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:29.114638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.66:50655)
0808 14:29:29.115036 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:29.115053 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:29.115164 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:29.115415 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:29.115417 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:29.115602 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:29.115695 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:29.115700 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:29.116444 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:29.116446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:29.117008 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:29.117016 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:29.117445 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:29.117447 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:29.117510 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:29.118085 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:29.118106 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:29.119974 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":106,"threads_started":1}
W20260808 14:29:29.120389 17295 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:29.114770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:50655 (local address 127.7.125.126:36447)
0808 14:29:29.114952 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:29.114957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:29.115133 (+ 176us) server_negotiation.cc:407] Connection header received
0808 14:29:29.115233 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:29.115235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:29.115278 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:29.115381 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:29.115794 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:29.116349 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:29.117108 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:29.117344 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:29.117425 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:29.120150 (+ 2725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.66:50655: BlockingRecv error: recv got EOF from 127.7.125.66:50655 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 08 14:29:29 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:29 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:29.318830 17297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:29.313406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.67:54423)
0808 14:29:29.313679 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:29.313693 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:29.313790 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:29.314030 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:29.314032 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:29.314220 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:29.314312 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:29.314317 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:29.315129 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:29.315132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:29.315704 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:29.315712 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:29.316228 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:29.316231 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:29.316296 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:29.316802 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:29.316813 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:29.318658 (+ 1845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":72,"threads_started":1}
W20260808 14:29:29.318962 17295 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:29.313490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:54423 (local address 127.7.125.126:36447)
0808 14:29:29.313667 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:29.313672 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:29.313763 (+ 91us) server_negotiation.cc:407] Connection header received
0808 14:29:29.313870 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:29.313872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:29.313921 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:29.313990 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:29.314410 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:29.315025 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:29.315844 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:29.316116 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:29.316199 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:29.318777 (+ 2578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.67:54423: BlockingRecv error: recv got EOF from 127.7.125.67:54423 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 08 14:29:29 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:29 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:29.935307 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:29.929640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.65:44353)
0808 14:29:29.930036 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:29.930070 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:29.930196 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:29.930509 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:29.930514 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:29.930816 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:29.930924 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:29.930931 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:29.931739 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:29.931741 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:29.932296 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:29.932303 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:29.932708 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:29.932710 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:29.932770 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:29.933251 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:29.933262 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:29.935137 (+ 1875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":126,"threads_started":1}
W20260808 14:29:29.935390 17298 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:29.929663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:44353 (local address 127.7.125.126:36447)
0808 14:29:29.930017 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:29.930023 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:29.930132 (+ 109us) server_negotiation.cc:407] Connection header received
0808 14:29:29.930302 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:29.930306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:29.930395 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:29.930470 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:29.931100 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:29.931642 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:29.932397 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:29.932612 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:29.932687 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:29.935257 (+ 2570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.65:44353: BlockingRecv error: recv got EOF from 127.7.125.65:44353 (error 108)
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":127,"threads_started":1}
Aug 08 14:29:30 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:30 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:30.126541 17300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:30.120841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.66:58009)
0808 14:29:30.121136 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:30.121149 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:30.121288 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:30.121590 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:30.121593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:30.121812 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:30.121905 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:30.121911 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:30.122674 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:30.122679 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:30.123331 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:30.123341 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:30.123813 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:30.123818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:30.123899 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:30.124468 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:30.124480 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:30.126366 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":90,"threads_started":1}
W20260808 14:29:30.126603 17298 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:30.120930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:58009 (local address 127.7.125.126:36447)
0808 14:29:30.121102 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:30.121107 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:30.121231 (+ 124us) server_negotiation.cc:407] Connection header received
0808 14:29:30.121419 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:30.121422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:30.121475 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:30.121571 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:30.122020 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:30.122562 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:30.123451 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:30.123697 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:30.123816 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:30.126495 (+ 2679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.66:58009: BlockingRecv error: recv got EOF from 127.7.125.66:58009 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 08 14:29:30 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:30 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:30.325168 17301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:30.319610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.67:48235)
0808 14:29:30.319913 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:30.319939 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:30.320078 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:30.320351 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:30.320354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:30.320577 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:30.320668 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:30.320673 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:30.321436 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:30.321443 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:30.322003 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:30.322012 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:30.322470 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:30.322474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:30.322563 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:30.323163 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:30.323175 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:30.325055 (+ 1880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":91,"threads_started":1}
W20260808 14:29:30.325350 17298 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:30.319679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:48235 (local address 127.7.125.126:36447)
0808 14:29:30.319849 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:30.319854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:30.320015 (+ 161us) server_negotiation.cc:407] Connection header received
0808 14:29:30.320180 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:30.320184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:30.320237 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:30.320325 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:30.320791 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:30.321317 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:30.322129 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:30.322355 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:30.322466 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:30.325194 (+ 2728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.67:48235: BlockingRecv error: recv got EOF from 127.7.125.67:48235 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 08 14:29:30 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:30 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:30.944957 17302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:30.936187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.65:47977)
0808 14:29:30.936541 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:30.936555 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:30.936650 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:30.936973 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:30.936978 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:30.937187 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:30.937286 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:30.937291 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:30.938378 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:30.938381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:30.939094 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:30.939104 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:30.939556 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:30.939558 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:30.939624 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:30.942928 (+ 3304us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:30.942940 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:30.944833 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":72,"threads_started":1}
W20260808 14:29:30.945161 17303 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:30.936269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:47977 (local address 127.7.125.126:36447)
0808 14:29:30.936602 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:30.936609 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:30.936630 (+ 21us) server_negotiation.cc:407] Connection header received
0808 14:29:30.936752 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:30.936755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:30.936812 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:30.936925 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:30.937445 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:30.938276 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:30.939218 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:30.939462 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:30.939547 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:30.944982 (+ 5435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.65:47977: BlockingRecv error: recv got EOF from 127.7.125.65:47977 (error 108)
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":133,"threads_started":1}
Aug 08 14:29:31 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:31 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:31.133337 17304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:31.127472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.66:43455)
0808 14:29:31.127836 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:31.127849 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:31.127982 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:31.128260 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:31.128264 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:31.128483 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:31.128667 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:31.128674 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:31.129505 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:31.129510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:31.130189 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:31.130198 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:31.130667 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:31.130671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:31.130757 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:31.131317 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:31.131329 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:31.133216 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":97,"threads_started":1}
W20260808 14:29:31.133531 17303 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:31.127589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:43455 (local address 127.7.125.126:36447)
0808 14:29:31.127741 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:31.127745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:31.127939 (+ 194us) server_negotiation.cc:407] Connection header received
0808 14:29:31.128101 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:31.128108 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:31.128153 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:31.128236 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:31.128805 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:31.129368 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:31.130310 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:31.130548 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:31.130662 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:31.133359 (+ 2697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.66:43455: BlockingRecv error: recv got EOF from 127.7.125.66:43455 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 14:29:31 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:31 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:31.331735 17305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:31.325959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.67:55603)
0808 14:29:31.326291 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:31.326305 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:31.326408 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:31.326634 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:31.326636 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:31.326834 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:31.327039 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:31.327055 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:31.327791 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:31.327794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:31.328428 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:31.328437 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:31.328870 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:31.328872 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:31.328931 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:31.329588 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:31.329600 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:31.331562 (+ 1962us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":105,"threads_started":1}
W20260808 14:29:31.331825 17303 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:31.326018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:55603 (local address 127.7.125.126:36447)
0808 14:29:31.326171 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:31.326176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:31.326382 (+ 206us) server_negotiation.cc:407] Connection header received
0808 14:29:31.326487 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:31.326489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:31.326538 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:31.326598 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:31.327166 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:31.327694 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:31.328531 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:31.328750 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:31.328832 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:31.331702 (+ 2870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.67:55603: BlockingRecv error: recv got EOF from 127.7.125.67:55603 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 14:29:31 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:31 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:31.951413 17306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:31.945804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.65:55747)
0808 14:29:31.946093 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:31.946107 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:31.946276 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:31.946641 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:31.946644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:31.946834 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:31.946929 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:31.946934 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:31.947702 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:31.947706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:31.948304 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:31.948313 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:31.948741 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:31.948744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:31.948815 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:31.949298 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:31.949310 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:31.951301 (+ 1991us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":111,"threads_started":1}
W20260808 14:29:31.951607 17307 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:31.945903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:55747 (local address 127.7.125.126:36447)
0808 14:29:31.946168 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:31.946174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:31.946199 (+ 25us) server_negotiation.cc:407] Connection header received
0808 14:29:31.946437 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:31.946440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:31.946503 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:31.946607 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:31.947060 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:31.947596 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:31.948416 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:31.948641 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:31.948720 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:31.951433 (+ 2713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.65:55747: BlockingRecv error: recv got EOF from 127.7.125.65:55747 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":100,"threads_started":1}
Aug 08 14:29:32 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:32 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:32.139757 17308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:32.134057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.66:60125)
0808 14:29:32.134332 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:32.134353 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:32.134454 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:32.134682 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:32.134684 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:32.134880 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:32.135001 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:32.135007 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:32.135876 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:32.135878 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:32.136421 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:32.136429 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:32.136847 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:32.136850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:32.136912 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:32.137517 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:32.137530 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:32.139627 (+ 2097us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":98,"threads_started":1}
W20260808 14:29:32.139904 17307 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:32.134147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:60125 (local address 127.7.125.126:36447)
0808 14:29:32.134293 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:32.134297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:32.134432 (+ 135us) server_negotiation.cc:407] Connection header received
0808 14:29:32.134527 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:32.134529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:32.134573 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:32.134643 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:32.135108 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:32.135768 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:32.136520 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:32.136735 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:32.136812 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:32.139751 (+ 2939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.66:60125: BlockingRecv error: recv got EOF from 127.7.125.66:60125 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 14:29:32 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:32 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:32.338280 17309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:32.332572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.67:44097)
0808 14:29:32.332866 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:32.332879 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:32.333019 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:32.333361 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:32.333364 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:32.333588 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:32.333681 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:32.333687 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:32.334486 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:32.334492 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:32.335128 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:32.335138 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:32.335597 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:32.335600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:32.335683 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:32.336188 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:32.336200 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:32.338094 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":110,"threads_started":1}
W20260808 14:29:32.338338 17307 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:32.332691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:44097 (local address 127.7.125.126:36447)
0808 14:29:32.332863 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:32.332868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:32.332969 (+ 101us) server_negotiation.cc:407] Connection header received
0808 14:29:32.333136 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:32.333139 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:32.333196 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:32.333336 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:32.333824 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:32.334366 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:32.335267 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:32.335482 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:32.335590 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:32.338221 (+ 2631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.67:44097: BlockingRecv error: recv got EOF from 127.7.125.67:44097 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 08 14:29:32 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:32 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:32.958076 17310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:32.952388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.65:41035)
0808 14:29:32.952685 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:32.952708 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:32.952805 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:32.953045 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:32.953047 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:32.953230 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:32.953324 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:32.953329 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:32.954059 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:32.954062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:32.954599 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:32.954605 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:32.955106 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:32.955109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:32.955172 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:32.955736 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:32.955748 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:32.957908 (+ 2160us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":78,"threads_started":1}
W20260808 14:29:32.958150 17311 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:32.952525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:41035 (local address 127.7.125.126:36447)
0808 14:29:32.952806 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:32.952828 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:32.952842 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:29:32.952882 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:32.952884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:32.952925 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:32.953004 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:32.953431 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:32.953968 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:32.954704 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:32.954934 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:32.955075 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:32.958028 (+ 2953us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.65:41035: BlockingRecv error: recv got EOF from 127.7.125.65:41035 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":83,"threads_started":1}
Aug 08 14:29:33 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:33 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:33.146551 17312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:33.140701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.66:35097)
0808 14:29:33.140990 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:33.141003 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:33.141144 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:33.141428 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:33.141431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:33.141644 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:33.141735 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:33.141740 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:33.142504 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:33.142508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:33.143118 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:33.143127 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:33.143580 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:33.143583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:33.143662 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:33.144247 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:33.144264 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:33.146416 (+ 2152us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":82,"threads_started":1}
W20260808 14:29:33.146705 17311 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:33.140753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:35097 (local address 127.7.125.126:36447)
0808 14:29:33.140925 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:33.140930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:33.141081 (+ 151us) server_negotiation.cc:407] Connection header received
0808 14:29:33.141245 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:33.141249 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:33.141302 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:33.141401 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:33.141859 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:33.142390 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:33.143234 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:33.143461 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:33.143574 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:33.146553 (+ 2979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.66:35097: BlockingRecv error: recv got EOF from 127.7.125.66:35097 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 08 14:29:33 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:33 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:33.345202 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:33.339141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.67:49623)
0808 14:29:33.339470 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:33.339483 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:33.339631 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:33.339937 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:33.339940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:33.340159 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:33.340250 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:33.340255 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:33.341028 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:33.341033 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:33.341598 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:33.341605 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:33.342072 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:33.342075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:33.342158 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:33.342734 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:33.342746 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:33.345063 (+ 2317us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":108,"threads_started":1}
W20260808 14:29:33.345340 17311 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:33.339183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:49623 (local address 127.7.125.126:36447)
0808 14:29:33.339334 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:33.339338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:33.339568 (+ 230us) server_negotiation.cc:407] Connection header received
0808 14:29:33.339745 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:33.339749 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:33.339806 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:33.339902 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:33.340373 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:33.340918 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:33.341720 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:33.341952 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:33.342056 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:33.345195 (+ 3139us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.67:49623: BlockingRecv error: recv got EOF from 127.7.125.67:49623 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 14:29:33 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:33 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:33.964676 17314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:33.959034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.65:37747)
0808 14:29:33.959316 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:33.959330 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:33.959415 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:33.959715 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:33.959717 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:33.959938 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:33.960036 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:33.960041 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:33.960790 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:33.960793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:33.961361 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:33.961368 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:33.961781 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:33.961783 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:33.961859 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:33.962345 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:33.962356 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:33.964502 (+ 2146us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":99,"threads_started":1}
W20260808 14:29:33.964792 17315 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:33.959173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:37747 (local address 127.7.125.126:36447)
0808 14:29:33.959482 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:33.959486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:33.959499 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:29:33.959537 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:33.959539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:33.959579 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:33.959662 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:33.960153 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:33.960694 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:33.961466 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:33.961682 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:33.961756 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:33.964651 (+ 2895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.65:37747: BlockingRecv error: recv got EOF from 127.7.125.65:37747 (error 108)
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":144,"threads_started":1}
Aug 08 14:29:34 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:34 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:34.153518 17316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:34.147494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.66:50347)
0808 14:29:34.147839 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:34.147856 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:34.148016 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:34.148303 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:34.148306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:34.148524 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:34.148617 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:34.148623 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:34.149401 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:34.149408 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:34.149975 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:34.149983 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:34.150462 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:34.150466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:34.150555 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:34.151176 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:34.151190 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:34.153372 (+ 2182us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":90,"threads_started":1}
W20260808 14:29:34.153741 17315 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:34.147631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:50347 (local address 127.7.125.126:36447)
0808 14:29:34.147818 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:34.147822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:34.147954 (+ 132us) server_negotiation.cc:407] Connection header received
0808 14:29:34.148121 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:34.148124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:34.148182 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:34.148275 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:34.148739 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:34.149281 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:34.150096 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:34.150334 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:34.150456 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:34.153514 (+ 3058us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.66:50347: BlockingRecv error: recv got EOF from 127.7.125.66:50347 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 08 14:29:34 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:34 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:34.352216 17317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:34.346058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.67:42749)
0808 14:29:34.346413 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:34.346429 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:34.346553 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:34.347344 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:34.347346 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:34.347530 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:34.347627 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:34.347632 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:34.348407 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:34.348409 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:34.348949 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:34.348956 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:34.349386 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:34.349389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:34.349447 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:34.350003 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:34.350014 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:34.352100 (+ 2086us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":114,"threads_started":1}
W20260808 14:29:34.352393 17315 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:34.346191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:42749 (local address 127.7.125.126:36447)
0808 14:29:34.347125 (+ 934us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:34.347129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:34.347143 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:29:34.347180 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:34.347182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:34.347227 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:34.347293 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:34.347760 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:34.348307 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:34.349055 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:34.349286 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:34.349369 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:34.352258 (+ 2889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.67:42749: BlockingRecv error: recv got EOF from 127.7.125.67:42749 (error 108)
Metrics: {"server-negotiator.queue_time_us":819}
Aug 08 14:29:34 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:34 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:34.970947 17318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:34.965690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.65:52923)
0808 14:29:34.966033 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:34.966047 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:34.966140 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:34.966407 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:34.966409 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:34.966591 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:34.966689 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:34.966694 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:34.967443 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:34.967446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:34.967996 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:34.968004 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:34.968430 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:34.968432 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:34.968494 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:34.968997 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:34.969009 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:34.970825 (+ 1816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":113,"threads_started":1}
W20260808 14:29:34.971161 17319 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:34.965853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:52923 (local address 127.7.125.126:36447)
0808 14:29:34.966086 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:34.966090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:34.966103 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:29:34.966232 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:34.966235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:34.966281 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:34.966357 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:34.966789 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:34.967345 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:34.968096 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:34.968326 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:34.968399 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:34.971010 (+ 2611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.65:52923: BlockingRecv error: recv got EOF from 127.7.125.65:52923 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":63,"threads_started":1}
Aug 08 14:29:35 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:35 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:35.160445 17320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:35.154326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.66:50313)
0808 14:29:35.154692 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:35.154706 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:35.154806 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:35.155200 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:35.155203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:35.155390 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:35.155485 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:35.155490 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:35.156203 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:35.156205 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:35.156813 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:35.156821 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:35.157580 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:35.157583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:35.157649 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:35.158178 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:35.158190 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:35.160278 (+ 2088us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":166,"threads_started":1}
W20260808 14:29:35.160619 17319 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:35.154415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:50313 (local address 127.7.125.126:36447)
0808 14:29:35.154649 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:35.154655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:35.154782 (+ 127us) server_negotiation.cc:407] Connection header received
0808 14:29:35.154891 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:35.154893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:35.154939 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:35.155059 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:35.155591 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:35.156110 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:35.157214 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:35.157453 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:35.157548 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:35.160475 (+ 2927us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.66:50313: BlockingRecv error: recv got EOF from 127.7.125.66:50313 (error 108)
Metrics: {"server-negotiator.queue_time_us":124}
Aug 08 14:29:35 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:35 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:35.359030 17321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:35.353154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.67:54247)
0808 14:29:35.353545 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:35.353557 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:35.353689 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:35.353960 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:35.353963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:35.354181 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:35.354280 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:35.354285 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:35.355208 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:35.355213 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:35.355765 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:35.355773 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:35.356252 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:35.356256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:35.356344 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:35.356927 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:35.356939 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:35.358823 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":131,"threads_started":1}
W20260808 14:29:35.359097 17319 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:35.353254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:54247 (local address 127.7.125.126:36447)
0808 14:29:35.353423 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:35.353429 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:35.353634 (+ 205us) server_negotiation.cc:407] Connection header received
0808 14:29:35.353791 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:35.353796 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:35.353853 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:35.353941 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:35.354423 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:35.355081 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:35.355899 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:35.356129 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:35.356236 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:35.358950 (+ 2714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.67:54247: BlockingRecv error: recv got EOF from 127.7.125.67:54247 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 08 14:29:35 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:35 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:35.981010 17322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:35.971916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.65:52491)
0808 14:29:35.972226 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:35.972238 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:35.972333 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:35.972526 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:35.972528 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:35.972707 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:35.972799 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:35.972804 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:35.973565 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:35.973568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:35.974151 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:35.974159 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:35.974586 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:35.974588 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:35.974650 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:35.978901 (+ 4251us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:35.978914 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:35.980836 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":107,"threads_started":1}
W20260808 14:29:35.981118 17323 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:35.971913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:52491 (local address 127.7.125.126:36447)
0808 14:29:35.972224 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:35.972228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:35.972289 (+ 61us) server_negotiation.cc:407] Connection header received
0808 14:29:35.972398 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:35.972400 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:35.972443 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:35.972515 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:35.972913 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:35.973468 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:35.974254 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:35.974488 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:35.974563 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:35.980977 (+ 6414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.65:52491: BlockingRecv error: recv got EOF from 127.7.125.65:52491 (error 108)
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":92,"threads_started":1}
Aug 08 14:29:36 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:36 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:36.166893 17324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:36.161327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.66:54025)
0808 14:29:36.161668 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:36.161681 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:36.161812 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:36.162094 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:36.162097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:36.162309 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:36.162418 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:36.162423 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:36.163261 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:36.163266 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:36.163827 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:36.163835 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:36.164317 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:36.164321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:36.164414 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:36.164917 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:36.164928 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:36.166762 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":112,"threads_started":1}
W20260808 14:29:36.167095 17323 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:36.161438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:54025 (local address 127.7.125.126:36447)
0808 14:29:36.161617 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:36.161622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:36.161753 (+ 131us) server_negotiation.cc:407] Connection header received
0808 14:29:36.161917 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:36.161920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:36.161976 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:36.162063 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:36.162538 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:36.163153 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:36.163954 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:36.164198 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:36.164311 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:36.166910 (+ 2599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.66:54025: BlockingRecv error: recv got EOF from 127.7.125.66:54025 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 08 14:29:36 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:36 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:36.365373 17325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:36.359869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.67:60755)
0808 14:29:36.360224 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:36.360236 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:36.360364 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:36.360635 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:36.360637 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:36.360828 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:36.360925 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:36.360931 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:36.361675 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:36.361683 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:36.362224 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:36.362233 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:36.362779 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:36.362785 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:36.362864 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:36.363424 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:36.363435 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:36.365261 (+ 1826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":114,"threads_started":1}
W20260808 14:29:36.365518 17323 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:36.360003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:60755 (local address 127.7.125.126:36447)
0808 14:29:36.360158 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:36.360162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:36.360314 (+ 152us) server_negotiation.cc:407] Connection header received
0808 14:29:36.360453 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:36.360455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:36.360505 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:36.360601 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:36.361034 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:36.361546 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:36.362354 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:36.362647 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:36.362737 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:36.365399 (+ 2662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.67:60755: BlockingRecv error: recv got EOF from 127.7.125.67:60755 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 14:29:36 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:36 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:36.987588 17326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:36.981918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.65:34149)
0808 14:29:36.982220 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:36.982235 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:36.982337 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:36.982606 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:36.982608 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:36.982795 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:36.982892 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:36.982897 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:36.983693 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:36.983699 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:36.984345 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:36.984355 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:36.984929 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:36.984936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:36.985029 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:36.985552 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:36.985563 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:36.987462 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":116,"threads_started":1}
W20260808 14:29:36.987795 17327 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:36.982018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:34149 (local address 127.7.125.126:36447)
0808 14:29:36.982270 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:36.982274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:36.982290 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:29:36.982429 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:36.982432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:36.982476 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:36.982567 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:36.983050 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:36.983577 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:36.984469 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:36.984773 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:36.984917 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:36.987620 (+ 2703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.65:34149: BlockingRecv error: recv got EOF from 127.7.125.65:34149 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":92,"threads_started":1}
Aug 08 14:29:37 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:37 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:37.173609 17328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:37.167875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.66:60741)
0808 14:29:37.168220 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:37.168236 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:37.168405 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:37.168697 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:37.168700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:37.168918 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:37.169011 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:37.169017 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:37.169805 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:37.169811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:37.170384 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:37.170400 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:37.170863 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:37.170867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:37.170957 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:37.171508 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:37.171520 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:37.173424 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":92,"threads_started":1}
W20260808 14:29:37.173657 17327 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:37.168043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:60741 (local address 127.7.125.126:36447)
0808 14:29:37.168224 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:37.168231 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:37.168326 (+ 95us) server_negotiation.cc:407] Connection header received
0808 14:29:37.168515 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:37.168518 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:37.168575 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:37.168668 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:37.169134 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:37.169692 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:37.170518 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:37.170746 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:37.170857 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:37.173548 (+ 2691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.66:60741: BlockingRecv error: recv got EOF from 127.7.125.66:60741 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 08 14:29:37 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:37 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:37.372290 17330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:37.366464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.67:57253)
0808 14:29:37.366733 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:37.366748 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:37.366917 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:37.367351 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:37.367354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:37.367571 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:37.367669 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:37.367674 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:37.368655 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:37.368658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:37.369221 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:37.369229 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:37.369658 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:37.369660 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:37.369720 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:37.370255 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:37.370267 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:37.372174 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":84,"threads_started":1}
W20260808 14:29:37.372500 17327 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:37.366521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:57253 (local address 127.7.125.126:36447)
0808 14:29:37.366714 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:37.366720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:37.366850 (+ 130us) server_negotiation.cc:407] Connection header received
0808 14:29:37.367095 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:37.367098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:37.367182 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:37.367276 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:37.367785 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:37.368534 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:37.369334 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:37.369556 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:37.369639 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:37.372356 (+ 2717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.67:57253: BlockingRecv error: recv got EOF from 127.7.125.67:57253 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 08 14:29:37 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:37 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:37.993708 17331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:37.988469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.65:34031)
0808 14:29:37.988783 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:37.988796 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:37.988887 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:37.989161 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:37.989163 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:37.989349 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:37.989446 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:37.989451 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:37.990230 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:37.990233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:37.990774 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:37.990781 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:37.991200 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:37.991203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:37.991264 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:37.991762 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:37.991774 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:37.993592 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":127,"threads_started":1}
W20260808 14:29:37.994153 17332 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:37.988529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:34031 (local address 127.7.125.126:36447)
0808 14:29:37.988825 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:37.988829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:37.988842 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:29:37.988993 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:37.988996 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:37.989040 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:37.989119 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:37.989545 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:37.990100 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:37.990871 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:37.991107 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:37.991180 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:37.993748 (+ 2568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.65:34031: BlockingRecv error: recv got EOF from 127.7.125.65:34031 (error 108)
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":85,"threads_started":1}
Aug 08 14:29:38 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:38 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:38.179930 17333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:38.174473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.66:52409)
0808 14:29:38.174765 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:38.174778 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:38.174877 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:38.175162 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:38.175164 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:38.175349 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:38.175446 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:38.175452 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:38.176163 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:38.176165 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:38.176730 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:38.176737 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:38.177167 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:38.177169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:38.177232 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:38.177787 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:38.177798 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:38.179783 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":120,"threads_started":1}
W20260808 14:29:38.180127 17332 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:38.174535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:52409 (local address 127.7.125.126:36447)
0808 14:29:38.174686 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:38.174689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:38.174856 (+ 167us) server_negotiation.cc:407] Connection header received
0808 14:29:38.174997 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:38.175000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:38.175046 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:38.175121 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:38.175551 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:38.176068 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:38.176846 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:38.177061 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:38.177138 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:38.179962 (+ 2824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.66:52409: BlockingRecv error: recv got EOF from 127.7.125.66:52409 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 14:29:38 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:38 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.67@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:38.378916 17334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:38.373202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.67:54151)
0808 14:29:38.373553 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:38.373568 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:38.373714 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:38.374069 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:38.374072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:38.374329 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:38.374436 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:38.374443 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:38.375211 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:38.375214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:38.375806 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:38.375836 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:38.376306 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:38.376309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:38.376379 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:38.376950 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:38.376963 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:38.378780 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":161,"threads_started":1}
W20260808 14:29:38.379123 17332 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:38.373283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:54151 (local address 127.7.125.126:36447)
0808 14:29:38.373446 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:38.373451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:38.373687 (+ 236us) server_negotiation.cc:407] Connection header received
0808 14:29:38.373867 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:38.373870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:38.373928 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:38.374016 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:38.374552 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:38.375107 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:38.375996 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:38.376212 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:38.376284 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:38.378932 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.67:54151: BlockingRecv error: recv got EOF from 127.7.125.67:54151 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 08 14:29:38 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:38 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.65@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:39.000730 17335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:38.994784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.65:52073)
0808 14:29:38.995134 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:38.995153 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:38.995267 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:38.995559 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:38.995562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:38.995775 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:38.995875 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:38.995881 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:38.996667 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:38.996672 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:38.997234 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:38.997242 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:38.997718 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:38.997721 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:38.997805 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:38.998382 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:38.998393 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:39.000602 (+ 2209us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":82,"threads_started":1}
W20260808 14:29:39.000931 17336 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:38.994954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:52073 (local address 127.7.125.126:36447)
0808 14:29:38.995285 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:38.995290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:38.995309 (+ 19us) server_negotiation.cc:407] Connection header received
0808 14:29:38.995372 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:38.995375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:38.995435 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:38.995534 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:38.996003 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:38.996551 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:38.997363 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:38.997598 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:38.997711 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:39.000735 (+ 3024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.65:52073: BlockingRecv error: recv got EOF from 127.7.125.65:52073 (error 108)
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":94,"threads_started":1}
Aug 08 14:29:39 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:29:39 dist-test-slave-w206 krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.125.66@KRBTEST.COM for oryx/127.7.125.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:29:39.186627 17337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:39.180889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36447 (local address 127.7.125.66:41441)
0808 14:29:39.181169 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:39.181189 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:39.181284 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:39.181530 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:39.181532 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:39.181719 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:39.181814 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:39.181820 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:39.182579 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:39.182583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:39.183184 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:39.183194 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:39.183645 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:39.183648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:39.183712 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:39.184333 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:39.184345 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:39.186449 (+ 2104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.125.126:36447: Server oryx/127.7.125.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":114,"threads_started":1}
W20260808 14:29:39.186730 17336 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:39.180935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:41441 (local address 127.7.125.126:36447)
0808 14:29:39.181091 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:39.181095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:39.181263 (+ 168us) server_negotiation.cc:407] Connection header received
0808 14:29:39.181375 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:39.181377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:39.181421 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:39.181499 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:39.181931 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:39.182448 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:39.183328 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:39.183541 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:39.183618 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:39.186613 (+ 2995us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.125.66:41441: BlockingRecv error: recv got EOF from 127.7.125.66:41441 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 14:29:39.269069 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 16844
I20260808 14:29:39.277537 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 17003
I20260808 14:29:39.285632 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 17140
I20260808 14:29:39.292019 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 16772
2026-08-08T14:29:39Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18501 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T14:29:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:29:39Z Disabled control of system clock
I20260808 14:29:39.326622 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:44215
--webserver_interface=127.7.125.126
--webserver_port=0
--builtin_ntp_servers=127.7.125.84:33793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:44215
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 14:29:39.438064 17341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:39.438325 17341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:39.438359 17341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:39.442351 17341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:29:39.442408 17341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:39.442425 17341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:39.442441 17341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:29:39.442457 17341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:29:39.447106 17341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:33793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:44215
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:44215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:39.448120 17341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:39.449409 17341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:39.458359 17346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:39.458343 17347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:39.458340 17341 server_base.cc:1034] running on GCE node
W20260808 14:29:39.458405 17349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:39.459334 17341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:39.460438 17341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:39.461598 17341 hybrid_clock.cc:648] HybridClock initialized: now 1786199379461571 us; error 48 us; skew 500 ppm
I20260808 14:29:39.463613 17341 webserver.cc:460] Webserver started at http://127.7.125.126:36891/ using document root <none> and password file <none>
I20260808 14:29:39.464148 17341 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:39.464208 17341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:39.464377 17341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:39.466348 17341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "0e6f5c28c93f4a4e9fb125859cb419b6"
format_stamp: "Formatted at 2026-08-08 14:29:39 on dist-test-slave-w206"
server_key: "c87ecd50ce70be2667587fd24b151c83"
server_key_iv: "6086c4b846507f00d3c410d31d2e0927"
server_key_version: "encryptionkey@0"
I20260808 14:29:39.466923 17341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "0e6f5c28c93f4a4e9fb125859cb419b6"
format_stamp: "Formatted at 2026-08-08 14:29:39 on dist-test-slave-w206"
server_key: "c87ecd50ce70be2667587fd24b151c83"
server_key_iv: "6086c4b846507f00d3c410d31d2e0927"
server_key_version: "encryptionkey@0"
I20260808 14:29:39.470618 17341 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:29:39.473256 17356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:39.474514 17341 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:29:39.474640 17341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "0e6f5c28c93f4a4e9fb125859cb419b6"
format_stamp: "Formatted at 2026-08-08 14:29:39 on dist-test-slave-w206"
server_key: "c87ecd50ce70be2667587fd24b151c83"
server_key_iv: "6086c4b846507f00d3c410d31d2e0927"
server_key_version: "encryptionkey@0"
I20260808 14:29:39.474754 17341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:39.532860 17341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:39.533627 17341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:39.533847 17341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:39.541596 17341 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:44215
I20260808 14:29:39.541611 17411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:44215 every 8 connection(s)
I20260808 14:29:39.542583 17341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:29:39.543648 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 17341
I20260808 14:29:39.543768 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:29:39.544121 7669 external_mini_cluster.cc:1428] Setting key e254e77ae45a940c4d7255f8613f36a9
I20260808 14:29:39.546216 17413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:39.551782 17413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e6f5c28c93f4a4e9fb125859cb419b6: Bootstrap starting.
I20260808 14:29:39.553439 17416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:39.545576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42604 (local address 127.7.125.126:44215)
0808 14:29:39.546237 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:39.546251 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:39.546320 (+ 69us) server_negotiation.cc:407] Connection header received
0808 14:29:39.547083 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:39.547102 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:39.547490 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:39.547836 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:39.548252 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:39.549211 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:39.550122 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:39.550586 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:39.550751 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:39.551674 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:39.551679 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:39.551691 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:29:39.552135 (+ 444us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:39.552193 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:29:39.552215 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:39.552326 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:39.552483 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 14:29:39.552736 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":159,"threads_started":1}
I20260808 14:29:39.554165 17413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e6f5c28c93f4a4e9fb125859cb419b6: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:39.555212 17413 log.cc:824] T 00000000000000000000000000000000 P 0e6f5c28c93f4a4e9fb125859cb419b6: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:39.557313 17413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e6f5c28c93f4a4e9fb125859cb419b6: No bootstrap required, opened a new log
I20260808 14:29:39.561185 17413 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e6f5c28c93f4a4e9fb125859cb419b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e6f5c28c93f4a4e9fb125859cb419b6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44215 } }
I20260808 14:29:39.561465 17413 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e6f5c28c93f4a4e9fb125859cb419b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:39.561532 17413 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e6f5c28c93f4a4e9fb125859cb419b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e6f5c28c93f4a4e9fb125859cb419b6, State: Initialized, Role: FOLLOWER
I20260808 14:29:39.562083 17413 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e6f5c28c93f4a4e9fb125859cb419b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e6f5c28c93f4a4e9fb125859cb419b6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44215 } }
I20260808 14:29:39.562235 17413 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e6f5c28c93f4a4e9fb125859cb419b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:29:39.562285 17413 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e6f5c28c93f4a4e9fb125859cb419b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:29:39.562357 17413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e6f5c28c93f4a4e9fb125859cb419b6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:39.563301 17413 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e6f5c28c93f4a4e9fb125859cb419b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e6f5c28c93f4a4e9fb125859cb419b6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44215 } }
I20260808 14:29:39.563601 17413 leader_election.cc:302] T 00000000000000000000000000000000 P 0e6f5c28c93f4a4e9fb125859cb419b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e6f5c28c93f4a4e9fb125859cb419b6; no voters:
I20260808 14:29:39.563856 17413 leader_election.cc:288] T 00000000000000000000000000000000 P 0e6f5c28c93f4a4e9fb125859cb419b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:29:39.564002 17418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e6f5c28c93f4a4e9fb125859cb419b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:29:39.564258 17418 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e6f5c28c93f4a4e9fb125859cb419b6 [term 1 LEADER]: Becoming Leader. State: Replica: 0e6f5c28c93f4a4e9fb125859cb419b6, State: Running, Role: LEADER
I20260808 14:29:39.564613 17418 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e6f5c28c93f4a4e9fb125859cb419b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e6f5c28c93f4a4e9fb125859cb419b6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44215 } }
I20260808 14:29:39.565119 17413 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e6f5c28c93f4a4e9fb125859cb419b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:29:39.566377 17420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e6f5c28c93f4a4e9fb125859cb419b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e6f5c28c93f4a4e9fb125859cb419b6. Latest consensus state: current_term: 1 leader_uuid: "0e6f5c28c93f4a4e9fb125859cb419b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e6f5c28c93f4a4e9fb125859cb419b6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44215 } } }
I20260808 14:29:39.566520 17420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e6f5c28c93f4a4e9fb125859cb419b6 [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:39.566811 17419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e6f5c28c93f4a4e9fb125859cb419b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e6f5c28c93f4a4e9fb125859cb419b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e6f5c28c93f4a4e9fb125859cb419b6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44215 } } }
I20260808 14:29:39.566922 17419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e6f5c28c93f4a4e9fb125859cb419b6 [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:39.566931 17428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:29:39.570058 17428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:29:39.575708 17428 catalog_manager.cc:1347] Generated new cluster ID: 788fe63068c44431baeea228b6222073
I20260808 14:29:39.575802 17428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:29:39.584213 17428 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:29:39.585117 17428 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:29:39.589742 17428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e6f5c28c93f4a4e9fb125859cb419b6: Generated new TSK 0
I20260808 14:29:39.590426 17428 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:29:39.599794 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:44215
--builtin_ntp_servers=127.7.125.84:33793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 14:29:39.714306 17438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:39.714583 17438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:39.714622 17438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:39.718720 17438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:39.718782 17438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:39.718837 17438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:29:39.723534 17438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:33793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:44215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:39.724587 17438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:39.725948 17438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:39.735698 17446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:39.735705 17443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:39.735858 17444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:39.735952 17438 server_base.cc:1034] running on GCE node
I20260808 14:29:39.736492 17438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:39.737144 17438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:39.738307 17438 hybrid_clock.cc:648] HybridClock initialized: now 1786199379738302 us; error 37 us; skew 500 ppm
I20260808 14:29:39.740191 17438 webserver.cc:460] Webserver started at http://127.7.125.65:40237/ using document root <none> and password file <none>
I20260808 14:29:39.740805 17438 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:39.740872 17438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:39.741137 17438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:39.743273 17438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "9af8fafd040645179de3231846e4f1f0"
format_stamp: "Formatted at 2026-08-08 14:29:39 on dist-test-slave-w206"
server_key: "1284ddc7645b2c053525f1236ddeb052"
server_key_iv: "9ca0263e3486c65dc69d4b49bca280e2"
server_key_version: "encryptionkey@0"
I20260808 14:29:39.743840 17438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "9af8fafd040645179de3231846e4f1f0"
format_stamp: "Formatted at 2026-08-08 14:29:39 on dist-test-slave-w206"
server_key: "1284ddc7645b2c053525f1236ddeb052"
server_key_iv: "9ca0263e3486c65dc69d4b49bca280e2"
server_key_version: "encryptionkey@0"
I20260808 14:29:39.747555 17438 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 14:29:39.750171 17452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:39.751472 17438 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:29:39.751613 17438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "9af8fafd040645179de3231846e4f1f0"
format_stamp: "Formatted at 2026-08-08 14:29:39 on dist-test-slave-w206"
server_key: "1284ddc7645b2c053525f1236ddeb052"
server_key_iv: "9ca0263e3486c65dc69d4b49bca280e2"
server_key_version: "encryptionkey@0"
I20260808 14:29:39.751729 17438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:39.774184 17438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:39.775038 17438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:39.775298 17438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:39.776013 17438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:39.777191 17438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:39.777274 17438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:39.777362 17438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:39.777426 17438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:39.788190 17438 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:42627
I20260808 14:29:39.788201 17571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:42627 every 8 connection(s)
I20260808 14:29:39.789232 17438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:29:39.797484 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 17438
I20260808 14:29:39.797631 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:29:39.797960 7669 external_mini_cluster.cc:1428] Setting key 38aef7ed4e71062f1f0fdb0947f49a78
I20260808 14:29:39.798656 17416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:39.791314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:44777 (local address 127.7.125.126:44215)
0808 14:29:39.791461 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:39.791465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:39.792563 (+ 1098us) server_negotiation.cc:407] Connection header received
0808 14:29:39.793874 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:39.793878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:39.793939 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:39.794032 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:39.794951 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:39.795541 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:39.796655 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:39.796867 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:39.796943 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:39.798129 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:39.798132 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:39.798136 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:29:39.798151 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:39.798176 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:29:39.798186 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:39.798265 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:39.798430 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 14:29:39.798509 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 14:29:39.799536 17574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:39.791715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:44215 (local address 127.7.125.65:44777)
0808 14:29:39.792385 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:39.792425 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:39.793659 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:39.794183 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:39.794189 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:39.794267 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:39.794738 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:39.794749 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:39.795674 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:39.795677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:39.796550 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:39.796559 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:39.797002 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:39.797005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:39.797142 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:39.797828 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:29:39.797829 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:39.797869 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:29:39.797932 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:39.798274 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:39.798282 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:39.798286 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 14:29:39.798421 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 14:29:39.798689 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":134,"threads_started":1}
I20260808 14:29:39.800068 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:44215
--builtin_ntp_servers=127.7.125.84:33793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 14:29:39.801244 17572 heartbeater.cc:347] Connected to a master server at 127.7.125.126:44215
I20260808 14:29:39.801487 17572 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:39.801997 17572 heartbeater.cc:511] Master 127.7.125.126:44215 requested a full tablet report, sending...
I20260808 14:29:39.803634 17375 ts_manager.cc:194] Registered new tserver with Master: 9af8fafd040645179de3231846e4f1f0 (127.7.125.65:42627)
I20260808 14:29:39.804845 17375 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.125.65:44777
W20260808 14:29:39.917917 17575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:39.918200 17575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:39.918241 17575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:39.922379 17575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:39.922437 17575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:39.922530 17575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:29:39.927399 17575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:33793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:44215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:39.928512 17575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:39.929942 17575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:29:39.940007 17575 server_base.cc:1034] running on GCE node
W20260808 14:29:39.940089 17582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:39.940318 17581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:39.940343 17584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:39.940757 17575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:39.941422 17575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:39.942610 17575 hybrid_clock.cc:648] HybridClock initialized: now 1786199379942599 us; error 28 us; skew 500 ppm
I20260808 14:29:39.944535 17575 webserver.cc:460] Webserver started at http://127.7.125.66:44569/ using document root <none> and password file <none>
I20260808 14:29:39.945116 17575 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:39.945215 17575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:39.945459 17575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:39.947602 17575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "fa13195665c540e8bed54b6714e0fbb2"
format_stamp: "Formatted at 2026-08-08 14:29:39 on dist-test-slave-w206"
server_key: "9171bc90f639454ae3741048de302447"
server_key_iv: "becd9cf6b2f39eb768a340bd89770819"
server_key_version: "encryptionkey@0"
I20260808 14:29:39.948174 17575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "fa13195665c540e8bed54b6714e0fbb2"
format_stamp: "Formatted at 2026-08-08 14:29:39 on dist-test-slave-w206"
server_key: "9171bc90f639454ae3741048de302447"
server_key_iv: "becd9cf6b2f39eb768a340bd89770819"
server_key_version: "encryptionkey@0"
I20260808 14:29:39.951840 17575 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:29:39.954319 17590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:39.955551 17575 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:29:39.955704 17575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "fa13195665c540e8bed54b6714e0fbb2"
format_stamp: "Formatted at 2026-08-08 14:29:39 on dist-test-slave-w206"
server_key: "9171bc90f639454ae3741048de302447"
server_key_iv: "becd9cf6b2f39eb768a340bd89770819"
server_key_version: "encryptionkey@0"
I20260808 14:29:39.955817 17575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:39.972623 17575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:39.973322 17575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:39.973544 17575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:39.974150 17575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:39.975155 17575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:39.975224 17575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:39.975291 17575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:39.975342 17575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:39.984668 17575 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:40623
I20260808 14:29:39.984690 17710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:40623 every 8 connection(s)
I20260808 14:29:39.985637 17575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:29:39.987810 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 17575
I20260808 14:29:39.987934 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:29:39.988286 7669 external_mini_cluster.cc:1428] Setting key bb5b96badc136f60c95e3a62f41a0e6d
I20260808 14:29:39.990238 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:44215
--builtin_ntp_servers=127.7.125.84:33793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 14:29:39.995529 17416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:39.987555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:44469 (local address 127.7.125.126:44215)
0808 14:29:39.987696 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:39.987700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:39.989039 (+ 1339us) server_negotiation.cc:407] Connection header received
0808 14:29:39.990548 (+ 1509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:39.990551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:39.990604 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:39.990683 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:39.991658 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:39.992187 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:39.993389 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:39.993619 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:39.993733 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:39.994954 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:39.994958 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:39.994960 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:29:39.995012 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:39.995038 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:29:39.995051 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:39.995132 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:39.995301 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 14:29:39.995377 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 14:29:39.996524 17713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:39.988153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:44215 (local address 127.7.125.66:44469)
0808 14:29:39.988880 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:39.988922 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:39.990254 (+ 1332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:39.990858 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:39.990866 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:39.990953 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:39.991506 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:39.991520 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:39.992336 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:39.992340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:39.993245 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:39.993255 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:39.993780 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:39.993785 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:39.993968 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:39.994676 (+ 708us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:29:39.994678 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:39.994720 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:29:39.994788 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:39.995134 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:39.995147 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:39.995153 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 14:29:39.995290 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 14:29:39.995605 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":164,"threads_started":1}
I20260808 14:29:39.997834 17711 heartbeater.cc:347] Connected to a master server at 127.7.125.126:44215
I20260808 14:29:39.998095 17711 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:39.998649 17711 heartbeater.cc:511] Master 127.7.125.126:44215 requested a full tablet report, sending...
I20260808 14:29:39.999859 17375 ts_manager.cc:194] Registered new tserver with Master: fa13195665c540e8bed54b6714e0fbb2 (127.7.125.66:40623)
I20260808 14:29:40.000788 17375 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.125.66:44469
W20260808 14:29:40.104722 17715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:40.104998 17715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:40.105034 17715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:40.109120 17715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:40.109195 17715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:40.109249 17715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:29:40.114112 17715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:33793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:44215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:40.115284 17715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:40.116660 17715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:40.126389 17723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:40.126407 17721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:40.126739 17720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:40.126950 17715 server_base.cc:1034] running on GCE node
I20260808 14:29:40.127490 17715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:40.128154 17715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:40.129348 17715 hybrid_clock.cc:648] HybridClock initialized: now 1786199380129335 us; error 25 us; skew 500 ppm
I20260808 14:29:40.131376 17715 webserver.cc:460] Webserver started at http://127.7.125.67:45319/ using document root <none> and password file <none>
I20260808 14:29:40.131984 17715 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:40.132084 17715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:40.132323 17715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:40.134658 17715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "4fd105d75e674d70993eb29b44c6ef4f"
format_stamp: "Formatted at 2026-08-08 14:29:40 on dist-test-slave-w206"
server_key: "e4e75ad0b2ec82d6a73f55f84682bfb2"
server_key_iv: "bb4bbff8ca5eb28859ea42116f481569"
server_key_version: "encryptionkey@0"
I20260808 14:29:40.135272 17715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "4fd105d75e674d70993eb29b44c6ef4f"
format_stamp: "Formatted at 2026-08-08 14:29:40 on dist-test-slave-w206"
server_key: "e4e75ad0b2ec82d6a73f55f84682bfb2"
server_key_iv: "bb4bbff8ca5eb28859ea42116f481569"
server_key_version: "encryptionkey@0"
I20260808 14:29:40.139067 17715 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 14:29:40.141633 17729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:40.142871 17715 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 14:29:40.143043 17715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "4fd105d75e674d70993eb29b44c6ef4f"
format_stamp: "Formatted at 2026-08-08 14:29:40 on dist-test-slave-w206"
server_key: "e4e75ad0b2ec82d6a73f55f84682bfb2"
server_key_iv: "bb4bbff8ca5eb28859ea42116f481569"
server_key_version: "encryptionkey@0"
I20260808 14:29:40.143168 17715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:40.164932 17715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:40.165792 17715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:40.166006 17715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:40.166697 17715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:40.167794 17715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:40.167843 17715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:40.167884 17715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:40.167914 17715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:40.176805 17715 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:43383
I20260808 14:29:40.176818 17846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:43383 every 8 connection(s)
I20260808 14:29:40.177771 17715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:29:40.187441 17416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:40.179878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:41545 (local address 127.7.125.126:44215)
0808 14:29:40.180060 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:40.180064 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:40.181186 (+ 1122us) server_negotiation.cc:407] Connection header received
0808 14:29:40.182649 (+ 1463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:40.182653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:40.182716 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:40.182837 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:40.183802 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:40.184345 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:40.185484 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:40.185711 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:40.185837 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:40.186910 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:40.186918 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:40.186920 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:29:40.186933 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:40.186951 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:29:40.186959 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:40.187064 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:40.187227 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 14:29:40.187290 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260808 14:29:40.187887 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 17715
I20260808 14:29:40.188009 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:29:40.188316 7669 external_mini_cluster.cc:1428] Setting key cecd70fa98c6a8fc8d157fd26ca89598
I20260808 14:29:40.188539 17849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:40.180334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:44215 (local address 127.7.125.67:41545)
0808 14:29:40.181021 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:40.181062 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:40.182413 (+ 1351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:40.183041 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:40.183048 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:40.183134 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:40.183632 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:40.183644 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:40.184481 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:40.184489 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:40.185352 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:40.185362 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:40.185870 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:40.185874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:40.186041 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:40.186657 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:29:40.186659 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:40.186696 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:29:40.186756 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:40.187078 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:40.187091 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:40.187101 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 14:29:40.187227 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 14:29:40.187503 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":118,"threads_started":1}
I20260808 14:29:40.189702 17847 heartbeater.cc:347] Connected to a master server at 127.7.125.126:44215
I20260808 14:29:40.189958 17847 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:40.190549 17847 heartbeater.cc:511] Master 127.7.125.126:44215 requested a full tablet report, sending...
I20260808 14:29:40.191622 17373 ts_manager.cc:194] Registered new tserver with Master: 4fd105d75e674d70993eb29b44c6ef4f (127.7.125.67:43383)
I20260808 14:29:40.192444 17373 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.125.67:41545
I20260808 14:29:40.202517 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 14:29:40.205425 17416 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:40.204480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42614 (local address 127.7.125.126:44215)
0808 14:29:40.204639 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:40.204645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:40.204786 (+ 141us) server_negotiation.cc:407] Connection header received
0808 14:29:40.204906 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:40.204908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:40.204949 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:40.205018 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:40.205290 (+ 272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42614: BlockingRecv error: recv got EOF from 127.0.0.1:42614 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 14:29:40.205713 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 17438
I20260808 14:29:40.212869 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 17575
I20260808 14:29:40.219350 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 17715
I20260808 14:29:40.225745 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 17341
2026-08-08T14:29:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (931 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T14:29:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:29:40Z Disabled control of system clock
I20260808 14:29:40.257522 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:38037
--webserver_interface=127.7.125.126
--webserver_port=0
--builtin_ntp_servers=127.7.125.84:46241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:38037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 14:29:40.369758 17860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:40.370076 17860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:40.370114 17860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:40.374142 17860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:29:40.374203 17860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:40.374218 17860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:40.374236 17860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:29:40.374253 17860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:29:40.378933 17860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:46241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:38037
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:38037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:40.380028 17860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:40.381379 17860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:40.389956 17869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:40.389997 17865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:40.389956 17866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:40.390197 17860 server_base.cc:1034] running on GCE node
I20260808 14:29:40.390801 17860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:40.391925 17860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:40.393110 17860 hybrid_clock.cc:648] HybridClock initialized: now 1786199380393082 us; error 44 us; skew 500 ppm
I20260808 14:29:40.395040 17860 webserver.cc:460] Webserver started at http://127.7.125.126:34367/ using document root <none> and password file <none>
I20260808 14:29:40.395615 17860 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:40.395709 17860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:40.395948 17860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:40.397981 17860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "d8f3495c90a44e92a7e52e069b58401f"
format_stamp: "Formatted at 2026-08-08 14:29:40 on dist-test-slave-w206"
server_key: "3999ccfe356165c5fb24fd4000303018"
server_key_iv: "d7786e9c80a6438bbaf5b51921e1435c"
server_key_version: "encryptionkey@0"
I20260808 14:29:40.398564 17860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "d8f3495c90a44e92a7e52e069b58401f"
format_stamp: "Formatted at 2026-08-08 14:29:40 on dist-test-slave-w206"
server_key: "3999ccfe356165c5fb24fd4000303018"
server_key_iv: "d7786e9c80a6438bbaf5b51921e1435c"
server_key_version: "encryptionkey@0"
I20260808 14:29:40.402299 17860 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 14:29:40.404808 17875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:40.406075 17860 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:29:40.406216 17860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "d8f3495c90a44e92a7e52e069b58401f"
format_stamp: "Formatted at 2026-08-08 14:29:40 on dist-test-slave-w206"
server_key: "3999ccfe356165c5fb24fd4000303018"
server_key_iv: "d7786e9c80a6438bbaf5b51921e1435c"
server_key_version: "encryptionkey@0"
I20260808 14:29:40.406365 17860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:40.411232 17860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:40.411886 17860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:40.412078 17860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:40.419271 17860 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:38037
I20260808 14:29:40.419269 17929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:38037 every 8 connection(s)
I20260808 14:29:40.420251 17860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:29:40.423661 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 17860
I20260808 14:29:40.423807 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:29:40.423925 17930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:40.424151 7669 external_mini_cluster.cc:1428] Setting key 13b3e6d41f4b4fefd10ed76a2a1a1a32
I20260808 14:29:40.429979 17933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:40.425562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50746 (local address 127.7.125.126:38037)
0808 14:29:40.426199 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:40.426213 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:40.426249 (+ 36us) server_negotiation.cc:407] Connection header received
0808 14:29:40.427169 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:40.427187 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:40.427536 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:40.427811 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:40.428688 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:40.428693 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:40.428705 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:29:40.428714 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:40.428767 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:29:40.428782 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:40.428893 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:40.429012 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 14:29:40.429274 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":148,"threads_started":1}
I20260808 14:29:40.430756 17930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8f3495c90a44e92a7e52e069b58401f: Bootstrap starting.
I20260808 14:29:40.432930 17930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8f3495c90a44e92a7e52e069b58401f: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:40.433670 17930 log.cc:824] T 00000000000000000000000000000000 P d8f3495c90a44e92a7e52e069b58401f: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:40.435602 17930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8f3495c90a44e92a7e52e069b58401f: No bootstrap required, opened a new log
I20260808 14:29:40.439088 17930 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8f3495c90a44e92a7e52e069b58401f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8f3495c90a44e92a7e52e069b58401f" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 38037 } }
I20260808 14:29:40.439312 17930 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8f3495c90a44e92a7e52e069b58401f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:40.439384 17930 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8f3495c90a44e92a7e52e069b58401f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8f3495c90a44e92a7e52e069b58401f, State: Initialized, Role: FOLLOWER
I20260808 14:29:40.439790 17930 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8f3495c90a44e92a7e52e069b58401f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8f3495c90a44e92a7e52e069b58401f" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 38037 } }
I20260808 14:29:40.439944 17930 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8f3495c90a44e92a7e52e069b58401f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:29:40.440011 17930 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8f3495c90a44e92a7e52e069b58401f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:29:40.440135 17930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8f3495c90a44e92a7e52e069b58401f [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:40.441047 17930 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8f3495c90a44e92a7e52e069b58401f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8f3495c90a44e92a7e52e069b58401f" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 38037 } }
I20260808 14:29:40.441383 17930 leader_election.cc:302] T 00000000000000000000000000000000 P d8f3495c90a44e92a7e52e069b58401f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8f3495c90a44e92a7e52e069b58401f; no voters:
I20260808 14:29:40.441650 17930 leader_election.cc:288] T 00000000000000000000000000000000 P d8f3495c90a44e92a7e52e069b58401f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:29:40.441799 17935 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8f3495c90a44e92a7e52e069b58401f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:29:40.442034 17935 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8f3495c90a44e92a7e52e069b58401f [term 1 LEADER]: Becoming Leader. State: Replica: d8f3495c90a44e92a7e52e069b58401f, State: Running, Role: LEADER
I20260808 14:29:40.442373 17935 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8f3495c90a44e92a7e52e069b58401f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8f3495c90a44e92a7e52e069b58401f" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 38037 } }
I20260808 14:29:40.443059 17930 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8f3495c90a44e92a7e52e069b58401f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:29:40.444133 17936 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8f3495c90a44e92a7e52e069b58401f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8f3495c90a44e92a7e52e069b58401f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8f3495c90a44e92a7e52e069b58401f" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 38037 } } }
I20260808 14:29:40.444276 17936 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8f3495c90a44e92a7e52e069b58401f [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:40.444584 17937 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8f3495c90a44e92a7e52e069b58401f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8f3495c90a44e92a7e52e069b58401f. Latest consensus state: current_term: 1 leader_uuid: "d8f3495c90a44e92a7e52e069b58401f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8f3495c90a44e92a7e52e069b58401f" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 38037 } } }
I20260808 14:29:40.444680 17937 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8f3495c90a44e92a7e52e069b58401f [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:40.444756 17944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:29:40.447985 17944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:29:40.453958 17944 catalog_manager.cc:1347] Generated new cluster ID: 8e137aaf61104b9daa27fb9a7e65987d
I20260808 14:29:40.454036 17944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:29:40.467526 17944 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:29:40.467873 17944 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:29:40.477183 17944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8f3495c90a44e92a7e52e069b58401f: Generated new TSK 0
I20260808 14:29:40.477897 17944 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:29:40.485913 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:38037
--builtin_ntp_servers=127.7.125.84:46241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 14:29:40.606107 17954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:40.606365 17954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:40.606406 17954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:40.610610 17954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:40.610664 17954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:40.610759 17954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:29:40.615677 17954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:46241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:38037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:40.616744 17954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:40.618144 17954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:40.628122 17961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:40.628264 17954 server_base.cc:1034] running on GCE node
W20260808 14:29:40.628134 17963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:40.628132 17960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:40.629055 17954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:40.629716 17954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:40.630877 17954 hybrid_clock.cc:648] HybridClock initialized: now 1786199380630848 us; error 54 us; skew 500 ppm
I20260808 14:29:40.632745 17954 webserver.cc:460] Webserver started at http://127.7.125.65:36085/ using document root <none> and password file <none>
I20260808 14:29:40.633330 17954 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:40.633394 17954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:40.633651 17954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:40.635821 17954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "9f242b7b64e643e0a1bafbf24ee980c8"
format_stamp: "Formatted at 2026-08-08 14:29:40 on dist-test-slave-w206"
server_key: "93ff5b7b4e802de36f41a586716d760f"
server_key_iv: "6364df9c4a0906f4f95ce83fca00e93e"
server_key_version: "encryptionkey@0"
I20260808 14:29:40.636381 17954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "9f242b7b64e643e0a1bafbf24ee980c8"
format_stamp: "Formatted at 2026-08-08 14:29:40 on dist-test-slave-w206"
server_key: "93ff5b7b4e802de36f41a586716d760f"
server_key_iv: "6364df9c4a0906f4f95ce83fca00e93e"
server_key_version: "encryptionkey@0"
I20260808 14:29:40.640090 17954 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 14:29:40.642490 17970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:40.643811 17954 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:29:40.643937 17954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "9f242b7b64e643e0a1bafbf24ee980c8"
format_stamp: "Formatted at 2026-08-08 14:29:40 on dist-test-slave-w206"
server_key: "93ff5b7b4e802de36f41a586716d760f"
server_key_iv: "6364df9c4a0906f4f95ce83fca00e93e"
server_key_version: "encryptionkey@0"
I20260808 14:29:40.644038 17954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:40.648572 17954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:40.649160 17954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:40.649345 17954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:40.649870 17954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:40.650777 17954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:40.650825 17954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:40.650890 17954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:40.650921 17954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:40.659381 17954 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:40509
I20260808 14:29:40.659405 18090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:40509 every 8 connection(s)
I20260808 14:29:40.660275 17954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:29:40.663903 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 17954
I20260808 14:29:40.664039 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:29:40.664330 7669 external_mini_cluster.cc:1428] Setting key b9d5715164aa07c9456b8fac5b475c25
I20260808 14:29:40.666544 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:38037
--builtin_ntp_servers=127.7.125.84:46241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 14:29:40.666635 17933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:40.662072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:50861 (local address 127.7.125.126:38037)
0808 14:29:40.662235 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:40.662239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:40.664023 (+ 1784us) server_negotiation.cc:407] Connection header received
0808 14:29:40.664795 (+ 772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:40.664799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:40.664843 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:40.664958 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:40.666170 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:40.666173 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:40.666175 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:29:40.666185 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:40.666205 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:29:40.666212 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:40.666286 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:40.666480 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 14:29:40.666515 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 14:29:40.667771 18093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:40.662452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:38037 (local address 127.7.125.65:50861)
0808 14:29:40.663141 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:40.663181 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:40.664583 (+ 1402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:40.665128 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:40.665134 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:40.665221 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:40.665878 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:29:40.665880 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:40.665933 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:29:40.666017 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:40.666305 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:40.666316 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:40.666325 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 14:29:40.666525 (+ 200us) client_negotiation.cc:240] Negotiation successful
0808 14:29:40.666771 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":120,"threads_started":1}
I20260808 14:29:40.668928 18091 heartbeater.cc:347] Connected to a master server at 127.7.125.126:38037
I20260808 14:29:40.669152 18091 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:40.669553 18091 heartbeater.cc:511] Master 127.7.125.126:38037 requested a full tablet report, sending...
I20260808 14:29:40.671049 17892 ts_manager.cc:194] Registered new tserver with Master: 9f242b7b64e643e0a1bafbf24ee980c8 (127.7.125.65:40509)
W20260808 14:29:40.780025 18094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:40.780292 18094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:40.780333 18094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:40.784582 18094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:40.784639 18094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:40.784698 18094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:29:40.789650 18094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:46241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:38037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:40.790719 18094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:40.792173 18094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:40.802079 18100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:40.802116 18102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:40.802102 18099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:40.802248 18094 server_base.cc:1034] running on GCE node
I20260808 14:29:40.802806 18094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:40.803512 18094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:40.804701 18094 hybrid_clock.cc:648] HybridClock initialized: now 1786199380804690 us; error 25 us; skew 500 ppm
I20260808 14:29:40.806602 18094 webserver.cc:460] Webserver started at http://127.7.125.66:43299/ using document root <none> and password file <none>
I20260808 14:29:40.807261 18094 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:40.807360 18094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:40.807603 18094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:40.809701 18094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "f9e084fe431940789814b981a20ad941"
format_stamp: "Formatted at 2026-08-08 14:29:40 on dist-test-slave-w206"
server_key: "97f265311d2c9ff35ff85795619bd3bf"
server_key_iv: "be7c7df6b976483c0dd1e0eb4333cf56"
server_key_version: "encryptionkey@0"
I20260808 14:29:40.810372 18094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "f9e084fe431940789814b981a20ad941"
format_stamp: "Formatted at 2026-08-08 14:29:40 on dist-test-slave-w206"
server_key: "97f265311d2c9ff35ff85795619bd3bf"
server_key_iv: "be7c7df6b976483c0dd1e0eb4333cf56"
server_key_version: "encryptionkey@0"
I20260808 14:29:40.814038 18094 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 14:29:40.816780 18108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:40.818027 18094 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:29:40.818164 18094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "f9e084fe431940789814b981a20ad941"
format_stamp: "Formatted at 2026-08-08 14:29:40 on dist-test-slave-w206"
server_key: "97f265311d2c9ff35ff85795619bd3bf"
server_key_iv: "be7c7df6b976483c0dd1e0eb4333cf56"
server_key_version: "encryptionkey@0"
I20260808 14:29:40.818284 18094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:40.822844 18094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:40.823544 18094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:40.823731 18094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:40.824330 18094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:40.825212 18094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:40.825284 18094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:40.825363 18094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:40.825398 18094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:40.833774 18094 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:40637
I20260808 14:29:40.833797 18226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:40637 every 8 connection(s)
I20260808 14:29:40.834652 18094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:29:40.840713 17933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:40.836558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:60449 (local address 127.7.125.126:38037)
0808 14:29:40.836699 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:40.836703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:40.837790 (+ 1087us) server_negotiation.cc:407] Connection header received
0808 14:29:40.839114 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:40.839117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:40.839156 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:40.839232 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:40.840352 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:40.840354 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:40.840356 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:29:40.840365 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:40.840380 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:29:40.840386 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:40.840461 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:40.840571 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 14:29:40.840607 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 14:29:40.841575 18229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:40.836953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:38037 (local address 127.7.125.66:60449)
0808 14:29:40.837637 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:40.837679 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:40.838849 (+ 1170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:40.839397 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:40.839404 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:40.839490 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:40.840117 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:29:40.840119 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:40.840159 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:29:40.840225 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:40.840461 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:40.840466 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:40.840469 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 14:29:40.840590 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 14:29:40.840823 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":109,"threads_started":1}
I20260808 14:29:40.842742 18227 heartbeater.cc:347] Connected to a master server at 127.7.125.126:38037
I20260808 14:29:40.843041 18227 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:40.843477 18227 heartbeater.cc:511] Master 127.7.125.126:38037 requested a full tablet report, sending...
I20260808 14:29:40.843879 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 18094
I20260808 14:29:40.844059 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:29:40.844357 7669 external_mini_cluster.cc:1428] Setting key bdd84f1b3706b5d975d27dbf4bb1f995
I20260808 14:29:40.845350 17891 ts_manager.cc:194] Registered new tserver with Master: f9e084fe431940789814b981a20ad941 (127.7.125.66:40637)
I20260808 14:29:40.846549 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:38037
--builtin_ntp_servers=127.7.125.84:46241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 14:29:40.958607 18230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:40.958850 18230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:40.958889 18230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:40.962903 18230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:40.963007 18230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:40.963091 18230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:29:40.967723 18230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:46241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:38037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:40.968708 18230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:40.970000 18230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:40.979486 18235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:40.979563 18236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:40.979681 18230 server_base.cc:1034] running on GCE node
W20260808 14:29:40.979548 18238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:40.980283 18230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:40.980927 18230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:40.982113 18230 hybrid_clock.cc:648] HybridClock initialized: now 1786199380982100 us; error 24 us; skew 500 ppm
I20260808 14:29:40.988134 18230 webserver.cc:460] Webserver started at http://127.7.125.67:34869/ using document root <none> and password file <none>
I20260808 14:29:40.988725 18230 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:40.988821 18230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:40.989060 18230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:40.991164 18230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "cf68bdacbfd64073b232465f92094e1d"
format_stamp: "Formatted at 2026-08-08 14:29:40 on dist-test-slave-w206"
server_key: "e59cc1d881e76078d99748d630628aa8"
server_key_iv: "992c6ad4bbcff1665f2e1430169ff7de"
server_key_version: "encryptionkey@0"
I20260808 14:29:40.991741 18230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "cf68bdacbfd64073b232465f92094e1d"
format_stamp: "Formatted at 2026-08-08 14:29:40 on dist-test-slave-w206"
server_key: "e59cc1d881e76078d99748d630628aa8"
server_key_iv: "992c6ad4bbcff1665f2e1430169ff7de"
server_key_version: "encryptionkey@0"
I20260808 14:29:40.995362 18230 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20260808 14:29:40.997987 18244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:40.999330 18230 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:29:40.999470 18230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "cf68bdacbfd64073b232465f92094e1d"
format_stamp: "Formatted at 2026-08-08 14:29:40 on dist-test-slave-w206"
server_key: "e59cc1d881e76078d99748d630628aa8"
server_key_iv: "992c6ad4bbcff1665f2e1430169ff7de"
server_key_version: "encryptionkey@0"
I20260808 14:29:40.999586 18230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:41.004288 18230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:41.004912 18230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:41.005103 18230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:41.005648 18230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:41.006572 18230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:41.006644 18230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:41.006712 18230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:41.006767 18230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:41.015671 18230 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:38315
I20260808 14:29:41.015688 18361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:38315 every 8 connection(s)
I20260808 14:29:41.016588 18230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:29:41.022917 17933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:41.018505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:53487 (local address 127.7.125.126:38037)
0808 14:29:41.018656 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:41.018660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:41.019793 (+ 1133us) server_negotiation.cc:407] Connection header received
0808 14:29:41.021130 (+ 1337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:41.021133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:41.021187 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:41.021273 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:41.022402 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:41.022405 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:41.022408 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:29:41.022421 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:41.022448 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:29:41.022457 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:41.022574 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:41.022725 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 14:29:41.022771 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 14:29:41.023746 18364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:41.018902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:38037 (local address 127.7.125.67:53487)
0808 14:29:41.019640 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:41.019680 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:41.020914 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:41.021491 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:41.021498 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:41.021586 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:41.022145 (+ 559us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:29:41.022147 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:41.022187 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:29:41.022254 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:41.022566 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:41.022575 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:41.022582 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 14:29:41.022756 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 14:29:41.023015 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":137,"threads_started":1}
I20260808 14:29:41.024044 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 18230
I20260808 14:29:41.024175 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:29:41.024530 7669 external_mini_cluster.cc:1428] Setting key cfb6ebf2abcd4a52f3bd62fc1a48a082
I20260808 14:29:41.024920 18362 heartbeater.cc:347] Connected to a master server at 127.7.125.126:38037
I20260808 14:29:41.025173 18362 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:41.025588 18362 heartbeater.cc:511] Master 127.7.125.126:38037 requested a full tablet report, sending...
I20260808 14:29:41.026640 17891 ts_manager.cc:194] Registered new tserver with Master: cf68bdacbfd64073b232465f92094e1d (127.7.125.67:38315)
I20260808 14:29:41.038487 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 14:29:41.041623 17933 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:29:41.040581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50754 (local address 127.7.125.126:38037)
0808 14:29:41.040755 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:41.040761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:41.040923 (+ 162us) server_negotiation.cc:407] Connection header received
0808 14:29:41.041080 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:41.041085 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:41.041125 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:41.041210 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:41.041484 (+ 274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50754: BlockingRecv error: recv got EOF from 127.0.0.1:50754 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 14:29:41.041793 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 17954
I20260808 14:29:41.049904 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 18094
I20260808 14:29:41.056946 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 18230
I20260808 14:29:41.064819 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 17860
2026-08-08T14:29:41Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (839 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:29:41 dist-test-slave-w206 krb5kdc[18373](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:29:41 dist-test-slave-w206 krb5kdc[18373](info): set up 2 sockets
Aug 08 14:29:41 dist-test-slave-w206 krb5kdc[18373](info): commencing operation
krb5kdc: starting...
W20260808 14:29:42.132318 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:29:42 dist-test-slave-w206 krb5kdc[18373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:29:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:29:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:29:42.268527 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:34199
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:45253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:34199
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:42.381773 18386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:42.382076 18386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:42.382118 18386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:42.386217 18386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:29:42.386283 18386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:42.386303 18386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:42.386320 18386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:29:42.386335 18386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:29:42.391669 18386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:45253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:34199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:34199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:42.392700 18386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:42.393685 18386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:42.402415 18391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:42.402416 18394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:42.402595 18386 server_base.cc:1034] running on GCE node
W20260808 14:29:42.402455 18392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:42.403368 18386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:42.404487 18386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:42.405676 18386 hybrid_clock.cc:648] HybridClock initialized: now 1786199382405652 us; error 43 us; skew 500 ppm
Aug 08 14:29:42 dist-test-slave-w206 krb5kdc[18373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199382, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:42.412972 18386 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:29:42.413960 18386 webserver.cc:460] Webserver started at http://127.7.125.126:42595/ using document root <none> and password file <none>
I20260808 14:29:42.414527 18386 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:42.414618 18386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:42.414849 18386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:42.417057 18386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "b538c01084014aa1a6633e1f7faf48ac"
format_stamp: "Formatted at 2026-08-08 14:29:42 on dist-test-slave-w206"
server_key: "7cfd1fda24447e96b38d67975d7287ea"
server_key_iv: "b5bd15e42ac81c918b044a6a2ad19c44"
server_key_version: "encryptionkey@0"
I20260808 14:29:42.417623 18386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "b538c01084014aa1a6633e1f7faf48ac"
format_stamp: "Formatted at 2026-08-08 14:29:42 on dist-test-slave-w206"
server_key: "7cfd1fda24447e96b38d67975d7287ea"
server_key_iv: "b5bd15e42ac81c918b044a6a2ad19c44"
server_key_version: "encryptionkey@0"
I20260808 14:29:42.421355 18386 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 14:29:42.423750 18402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:42.424986 18386 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:29:42.425124 18386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "b538c01084014aa1a6633e1f7faf48ac"
format_stamp: "Formatted at 2026-08-08 14:29:42 on dist-test-slave-w206"
server_key: "7cfd1fda24447e96b38d67975d7287ea"
server_key_iv: "b5bd15e42ac81c918b044a6a2ad19c44"
server_key_version: "encryptionkey@0"
I20260808 14:29:42.425231 18386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:42.442739 18386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:42.446002 18386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:42.446211 18386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:42.453853 18386 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:34199
I20260808 14:29:42.454166 18457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:34199 every 8 connection(s)
I20260808 14:29:42.454877 18386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:29:42.456001 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 18386
I20260808 14:29:42.456139 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:29:42.456467 7669 external_mini_cluster.cc:1428] Setting key 56d735f00e6e54bc99a74dbd7758adc0
I20260808 14:29:42.458244 18458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:42.464820 18458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac: Bootstrap starting.
Aug 08 14:29:42 dist-test-slave-w206 krb5kdc[18373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:29:42.467466 18458 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:42.468366 18458 log.cc:824] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:42.470741 18458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac: No bootstrap required, opened a new log
I20260808 14:29:42.471464 18461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:42.457851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51604 (local address 127.7.125.126:34199)
0808 14:29:42.458279 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:42.458295 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:42.458329 (+ 34us) server_negotiation.cc:407] Connection header received
0808 14:29:42.459085 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:42.459103 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:42.459420 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:42.459702 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:42.460626 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:42.461605 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:42.462744 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:42.463281 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:42.463395 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:42.466153 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:42.466160 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:42.466176 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:42.466207 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:42.468209 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:42.468680 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:42.468686 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:42.468691 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:42.468835 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:42.469160 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:42.469163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:42.469164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:42.469700 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:42.469880 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:42.470157 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 14:29:42.470460 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":138,"threads_started":1}
I20260808 14:29:42.474738 18458 raft_consensus.cc:348] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b538c01084014aa1a6633e1f7faf48ac" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 34199 } }
I20260808 14:29:42.474953 18458 raft_consensus.cc:374] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:42.475054 18458 raft_consensus.cc:729] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b538c01084014aa1a6633e1f7faf48ac, State: Initialized, Role: FOLLOWER
I20260808 14:29:42.475463 18458 consensus_queue.cc:260] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b538c01084014aa1a6633e1f7faf48ac" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 34199 } }
I20260808 14:29:42.475584 18458 raft_consensus.cc:388] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:29:42.475629 18458 raft_consensus.cc:482] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:29:42.475698 18458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:42.476526 18458 raft_consensus.cc:504] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b538c01084014aa1a6633e1f7faf48ac" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 34199 } }
I20260808 14:29:42.476867 18458 leader_election.cc:302] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b538c01084014aa1a6633e1f7faf48ac; no voters:
I20260808 14:29:42.477154 18458 leader_election.cc:288] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:29:42.477283 18463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:29:42.477548 18463 raft_consensus.cc:686] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac [term 1 LEADER]: Becoming Leader. State: Replica: b538c01084014aa1a6633e1f7faf48ac, State: Running, Role: LEADER
I20260808 14:29:42.477923 18463 consensus_queue.cc:237] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b538c01084014aa1a6633e1f7faf48ac" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 34199 } }
I20260808 14:29:42.478459 18458 sys_catalog.cc:564] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:29:42.479630 18464 sys_catalog.cc:455] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b538c01084014aa1a6633e1f7faf48ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b538c01084014aa1a6633e1f7faf48ac" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 34199 } } }
I20260808 14:29:42.479777 18464 sys_catalog.cc:458] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:42.480096 18465 sys_catalog.cc:455] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader b538c01084014aa1a6633e1f7faf48ac. Latest consensus state: current_term: 1 leader_uuid: "b538c01084014aa1a6633e1f7faf48ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b538c01084014aa1a6633e1f7faf48ac" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 34199 } } }
I20260808 14:29:42.480195 18465 sys_catalog.cc:458] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac [sys.catalog]: This master's current role is: LEADER
I20260808 14:29:42.480187 18473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:29:42.483599 18473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:29:42.489187 18473 catalog_manager.cc:1347] Generated new cluster ID: 3b34b006430c42c7adcca0d42f052f5f
I20260808 14:29:42.489248 18473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:29:42.504837 18473 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:29:42.505764 18473 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:29:42.510392 18473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac: Generated new TSK 0
I20260808 14:29:42.511128 18473 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.125.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
I20260808 14:29:42.542110 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:34199
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:45253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:42.687266 18485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:42.687529 18485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:42.687601 18485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:42.691753 18485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:42.691817 18485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:42.691908 18485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:29:42.697078 18485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:45253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:34199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:42.698154 18485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:42.699299 18485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:42.708992 18491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:42.709136 18490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:42.709088 18494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:42.709244 18485 server_base.cc:1034] running on GCE node
I20260808 14:29:42.709617 18485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:42.710276 18485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:42.711460 18485 hybrid_clock.cc:648] HybridClock initialized: now 1786199382711440 us; error 33 us; skew 500 ppm
Aug 08 14:29:42 dist-test-slave-w206 krb5kdc[18373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199382, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:42.714130 18485 init.cc:377] Logged in from keytab as kudu/127.7.125.65@KRBTEST.COM (short username kudu)
I20260808 14:29:42.714949 18485 webserver.cc:460] Webserver started at http://127.7.125.65:33785/ using document root <none> and password file <none>
I20260808 14:29:42.715555 18485 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:42.715648 18485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:42.715894 18485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:42.717898 18485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "4e5db532e1024c2886c56fca7f74af59"
format_stamp: "Formatted at 2026-08-08 14:29:42 on dist-test-slave-w206"
server_key: "bbfa1eac1087bbc6a39c37120ce1b2d0"
server_key_iv: "39c3c53ba210bcbcf2586bb80422a16e"
server_key_version: "encryptionkey@0"
I20260808 14:29:42.718461 18485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "4e5db532e1024c2886c56fca7f74af59"
format_stamp: "Formatted at 2026-08-08 14:29:42 on dist-test-slave-w206"
server_key: "bbfa1eac1087bbc6a39c37120ce1b2d0"
server_key_iv: "39c3c53ba210bcbcf2586bb80422a16e"
server_key_version: "encryptionkey@0"
I20260808 14:29:42.722121 18485 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 14:29:42.724493 18501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:42.725678 18485 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:29:42.725831 18485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "4e5db532e1024c2886c56fca7f74af59"
format_stamp: "Formatted at 2026-08-08 14:29:42 on dist-test-slave-w206"
server_key: "bbfa1eac1087bbc6a39c37120ce1b2d0"
server_key_iv: "39c3c53ba210bcbcf2586bb80422a16e"
server_key_version: "encryptionkey@0"
I20260808 14:29:42.725942 18485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:42.737210 18485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:42.740304 18485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:42.740530 18485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:42.741214 18485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:42.742056 18485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:42.742127 18485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:42.742190 18485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:42.742241 18485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:42.751180 18485 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:34713
I20260808 14:29:42.751250 18621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:34713 every 8 connection(s)
I20260808 14:29:42.752081 18485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:29:42.760089 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 18485
I20260808 14:29:42.760233 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:29:42.760571 7669 external_mini_cluster.cc:1428] Setting key 91d034863aad91ec89b61d3826cb98fa
Aug 08 14:29:42 dist-test-slave-w206 krb5kdc[18373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199382, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:29:42.766049 18461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:42.753851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:49779 (local address 127.7.125.126:34199)
0808 14:29:42.753999 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:42.754003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:42.754748 (+ 745us) server_negotiation.cc:407] Connection header received
0808 14:29:42.756170 (+ 1422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:42.756174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:42.756234 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:42.756321 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:42.757434 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:42.757980 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:42.759121 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:42.759392 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:42.759511 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:42.762450 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:42.762454 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:42.762456 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:42.762480 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:42.763995 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:42.764528 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:42.764536 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:42.764538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:42.764613 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:42.764959 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:42.764964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:42.764966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:42.765143 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:42.765249 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:42.765730 (+ 481us) server_negotiation.cc:299] Negotiation successful
0808 14:29:42.765874 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:29:42.766920 18624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:42.754155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:34199 (local address 127.7.125.65:49779)
0808 14:29:42.754590 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:42.754632 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:42.755956 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:42.756475 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:42.756482 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:42.756831 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:42.757280 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:42.757292 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:42.758109 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:42.758112 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:42.758992 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:42.759002 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:42.759552 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:42.759556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:42.759730 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:42.760475 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:42.760491 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:42.762274 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:42.764115 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:42.764121 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:42.764136 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:42.764410 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:42.764706 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:42.764710 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:42.764712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:42.764860 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:42.765251 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:42.765260 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:42.765501 (+ 241us) client_negotiation.cc:769] Sending connection context
0808 14:29:42.765690 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 14:29:42.766008 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":121,"threads_started":1}
I20260808 14:29:42.768477 18622 heartbeater.cc:347] Connected to a master server at 127.7.125.126:34199
I20260808 14:29:42.768738 18622 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:42.769225 18622 heartbeater.cc:511] Master 127.7.125.126:34199 requested a full tablet report, sending...
I20260808 14:29:42.770942 18419 ts_manager.cc:194] Registered new tserver with Master: 4e5db532e1024c2886c56fca7f74af59 (127.7.125.65:34713)
WARNING: no policy specified for kudu/127.7.125.66@KRBTEST.COM; defaulting to no policy
I20260808 14:29:42.772521 18419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:49779
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
I20260808 14:29:42.792057 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:34199
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:45253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:42.902503 18627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:42.902778 18627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:42.902832 18627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:42.906831 18627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:42.906889 18627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:42.906987 18627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:29:42.912287 18627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:45253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:34199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:42.913403 18627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:42.914472 18627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:42.924271 18634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:42.924360 18636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:42.924269 18633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:42.924590 18627 server_base.cc:1034] running on GCE node
I20260808 14:29:42.924998 18627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:42.925684 18627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:42.926852 18627 hybrid_clock.cc:648] HybridClock initialized: now 1786199382926839 us; error 24 us; skew 500 ppm
Aug 08 14:29:42 dist-test-slave-w206 krb5kdc[18373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199382, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:42.929952 18627 init.cc:377] Logged in from keytab as kudu/127.7.125.66@KRBTEST.COM (short username kudu)
I20260808 14:29:42.930824 18627 webserver.cc:460] Webserver started at http://127.7.125.66:41895/ using document root <none> and password file <none>
I20260808 14:29:42.931424 18627 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:42.931488 18627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:42.931653 18627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:42.933657 18627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "f1f68b508deb4840aa5f41defb38f628"
format_stamp: "Formatted at 2026-08-08 14:29:42 on dist-test-slave-w206"
server_key: "9cac72f21d896f553e03edb127701997"
server_key_iv: "0355f84072b7fce765389e76ccd4111a"
server_key_version: "encryptionkey@0"
I20260808 14:29:42.934172 18627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "f1f68b508deb4840aa5f41defb38f628"
format_stamp: "Formatted at 2026-08-08 14:29:42 on dist-test-slave-w206"
server_key: "9cac72f21d896f553e03edb127701997"
server_key_iv: "0355f84072b7fce765389e76ccd4111a"
server_key_version: "encryptionkey@0"
I20260808 14:29:42.937865 18627 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 14:29:42.940382 18645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:42.941711 18627 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:29:42.941807 18627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "f1f68b508deb4840aa5f41defb38f628"
format_stamp: "Formatted at 2026-08-08 14:29:42 on dist-test-slave-w206"
server_key: "9cac72f21d896f553e03edb127701997"
server_key_iv: "0355f84072b7fce765389e76ccd4111a"
server_key_version: "encryptionkey@0"
I20260808 14:29:42.941888 18627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:42.960062 18627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:42.963482 18627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:42.963639 18627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:42.964167 18627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:42.964949 18627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:42.964998 18627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:42.965035 18627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:42.965054 18627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:42.974090 18627 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:33397
I20260808 14:29:42.974115 18767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:33397 every 8 connection(s)
I20260808 14:29:42.974958 18627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:29:42.979168 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 18627
I20260808 14:29:42.979280 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:29:42.979626 7669 external_mini_cluster.cc:1428] Setting key b68658d837a3457f1429c79b0d5a33bd
Aug 08 14:29:42 dist-test-slave-w206 krb5kdc[18373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199382, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.125.67@KRBTEST.COM; defaulting to no policy
I20260808 14:29:42.989807 18461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:42.976837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:51241 (local address 127.7.125.126:34199)
0808 14:29:42.976979 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:42.976983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:42.977779 (+ 796us) server_negotiation.cc:407] Connection header received
0808 14:29:42.979292 (+ 1513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:42.979295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:42.979353 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:42.979436 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:42.981026 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:42.981546 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:42.982667 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:42.982956 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:42.983091 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:42.985979 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:42.985983 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:42.985986 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:42.986019 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:42.987629 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:42.988136 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:42.988141 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:42.988142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:42.988244 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:42.988606 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:42.988610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:42.988612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:42.988801 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:42.988922 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:42.989486 (+ 564us) server_negotiation.cc:299] Negotiation successful
0808 14:29:42.989637 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:29:42.990669 18770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:42.977195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:34199 (local address 127.7.125.66:51241)
0808 14:29:42.977640 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:42.977682 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:42.979087 (+ 1405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:42.979753 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:42.979763 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:42.980292 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:42.980886 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:42.980901 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:42.981675 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:42.981683 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:42.982542 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:42.982551 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:42.983276 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:42.983282 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:42.983515 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:42.984231 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:42.984247 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:42.985794 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:42.987743 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:42.987748 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:42.987763 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:42.988029 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:42.988342 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:42.988357 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:42.988359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:42.988505 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:42.988927 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:42.988935 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:42.989214 (+ 279us) client_negotiation.cc:769] Sending connection context
0808 14:29:42.989422 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 14:29:42.989779 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":129,"threads_started":1}
I20260808 14:29:42.991914 18768 heartbeater.cc:347] Connected to a master server at 127.7.125.126:34199
I20260808 14:29:42.992162 18768 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.67@KRBTEST.COM" created.
I20260808 14:29:42.992722 18768 heartbeater.cc:511] Master 127.7.125.126:34199 requested a full tablet report, sending...
I20260808 14:29:42.993902 18419 ts_manager.cc:194] Registered new tserver with Master: f1f68b508deb4840aa5f41defb38f628 (127.7.125.66:33397)
I20260808 14:29:42.994484 18419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:51241
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
I20260808 14:29:43.009699 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:34199
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:45253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:29:43.121986 18773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:29:43.122238 18773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:29:43.122282 18773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:29:43.126331 18773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:29:43.126390 18773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:29:43.126478 18773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:29:43.132069 18773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:45253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:34199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:29:43.133107 18773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:29:43.134115 18773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:29:43.143821 18782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:43.143822 18778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:29:43.143890 18779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:29:43.144070 18773 server_base.cc:1034] running on GCE node
I20260808 14:29:43.144632 18773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:29:43.145327 18773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:29:43.146539 18773 hybrid_clock.cc:648] HybridClock initialized: now 1786199383146511 us; error 40 us; skew 500 ppm
Aug 08 14:29:43 dist-test-slave-w206 krb5kdc[18373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199383, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:29:43.149449 18773 init.cc:377] Logged in from keytab as kudu/127.7.125.67@KRBTEST.COM (short username kudu)
I20260808 14:29:43.150344 18773 webserver.cc:460] Webserver started at http://127.7.125.67:45993/ using document root <none> and password file <none>
I20260808 14:29:43.150936 18773 fs_manager.cc:359] Metadata directory not provided
I20260808 14:29:43.151057 18773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:29:43.151304 18773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:29:43.153406 18773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "9eff95b97a89456da07b4786c2eafd1d"
format_stamp: "Formatted at 2026-08-08 14:29:43 on dist-test-slave-w206"
server_key: "4db180ec4af8ae6e028b5d2a3369708d"
server_key_iv: "71daccf2913e3d1fe15105301e013f99"
server_key_version: "encryptionkey@0"
I20260808 14:29:43.154011 18773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "9eff95b97a89456da07b4786c2eafd1d"
format_stamp: "Formatted at 2026-08-08 14:29:43 on dist-test-slave-w206"
server_key: "4db180ec4af8ae6e028b5d2a3369708d"
server_key_iv: "71daccf2913e3d1fe15105301e013f99"
server_key_version: "encryptionkey@0"
I20260808 14:29:43.157943 18773 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260808 14:29:43.160589 18789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:43.162350 18773 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 14:29:43.162487 18773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "9eff95b97a89456da07b4786c2eafd1d"
format_stamp: "Formatted at 2026-08-08 14:29:43 on dist-test-slave-w206"
server_key: "4db180ec4af8ae6e028b5d2a3369708d"
server_key_iv: "71daccf2913e3d1fe15105301e013f99"
server_key_version: "encryptionkey@0"
I20260808 14:29:43.162606 18773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:29:43.180552 18773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:29:43.184038 18773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:29:43.184257 18773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:29:43.184885 18773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:29:43.185726 18773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:29:43.185772 18773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:43.185845 18773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:29:43.185883 18773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:29:43.194644 18773 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:35507
I20260808 14:29:43.194721 18906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:35507 every 8 connection(s)
I20260808 14:29:43.195614 18773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:29:43.196878 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 18773
I20260808 14:29:43.197022 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:29:43.197324 7669 external_mini_cluster.cc:1428] Setting key 679baac660d2844428a1770019435aa7
Aug 08 14:29:43 dist-test-slave-w206 krb5kdc[18373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199383, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:29:43.209645 18461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:43.197621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:33705 (local address 127.7.125.126:34199)
0808 14:29:43.197764 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:43.197769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:43.198786 (+ 1017us) server_negotiation.cc:407] Connection header received
0808 14:29:43.200465 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:43.200468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:43.200524 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:43.200603 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:43.201757 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.202285 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.203534 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.203757 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.203871 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:43.206368 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:43.206371 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:43.206373 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:43.206398 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:43.207939 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:43.208386 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:43.208388 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:43.208390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:43.208460 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:43.208745 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:43.208747 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:43.208748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:43.208915 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:43.208988 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:43.209433 (+ 445us) server_negotiation.cc:299] Negotiation successful
0808 14:29:43.209524 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 14:29:43.210654 18909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:43.198050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:34199 (local address 127.7.125.67:33705)
0808 14:29:43.198639 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:43.198696 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:43.200249 (+ 1553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:43.200778 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:43.200785 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:43.201145 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:29:43.201614 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:43.201626 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.202420 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.202424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:43.203407 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:43.203417 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.203911 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.203915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:43.204100 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:43.204735 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:29:43.204751 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:29:43.206185 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:29:43.208039 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:43.208043 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:43.208055 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:43.208299 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:43.208538 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:29:43.208540 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:29:43.208542 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:29:43.208668 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:29:43.209001 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:29:43.209006 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:29:43.209231 (+ 225us) client_negotiation.cc:769] Sending connection context
0808 14:29:43.209385 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 14:29:43.209634 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":139,"threads_started":1}
I20260808 14:29:43.211910 18907 heartbeater.cc:347] Connected to a master server at 127.7.125.126:34199
I20260808 14:29:43.212117 18907 heartbeater.cc:464] Registering TS with master...
I20260808 14:29:43.212566 18907 heartbeater.cc:511] Master 127.7.125.126:34199 requested a full tablet report, sending...
I20260808 14:29:43.213527 18419 ts_manager.cc:194] Registered new tserver with Master: 9eff95b97a89456da07b4786c2eafd1d (127.7.125.67:35507)
I20260808 14:29:43.214111 18419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:33705
I20260808 14:29:43.223572 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:29:43.233055 18461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:43.225508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51616 (local address 127.7.125.126:34199)
0808 14:29:43.225611 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:43.225615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:43.225896 (+ 281us) server_negotiation.cc:407] Connection header received
0808 14:29:43.226056 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:43.226061 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:43.226107 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:43.226191 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:29:43.226910 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.227490 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.228337 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.228587 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.228700 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:43.230229 (+ 1529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:29:43.230232 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:29:43.230234 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:29:43.230257 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:29:43.231644 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:43.232031 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:43.232033 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:43.232034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:43.232105 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:29:43.232342 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:29:43.232344 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:29:43.232345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:29:43.232551 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:29:43.232652 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:43.232846 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 14:29:43.232938 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 14:29:43.235760 18419 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51616:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:29:43.238718 18419 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:29:43.253142 18919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:43.247488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:33397 (local address 127.0.0.1:40626)
0808 14:29:43.248329 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:43.248341 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:43.248432 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:43.248919 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:43.248925 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:43.248963 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:43.249138 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:43.249144 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.250240 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.250245 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:43.251551 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:43.251562 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.252610 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.252613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:43.252715 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:43.252750 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 14:29:43.252856 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 14:29:43.252972 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"mutex_wait_us":4,"spinlock_wait_cycles":3328,"thread_start_us":93,"threads_started":1}
I20260808 14:29:43.253353 18922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:43.247913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45106 (local address 127.7.125.65:34713)
0808 14:29:43.248798 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:43.248813 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:43.248835 (+ 22us) server_negotiation.cc:407] Connection header received
0808 14:29:43.248910 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:43.248915 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:43.249085 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:43.249242 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:43.249463 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.250534 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.251958 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.252687 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.252839 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:43.252979 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:43.253075 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 14:29:43.253188 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":782,"thread_start_us":96,"threads_started":1}
I20260808 14:29:43.253561 18921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:43.247194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:34713 (local address 127.0.0.1:45106)
0808 14:29:43.248049 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:43.248062 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:43.248128 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:43.249231 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:43.249234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:43.249254 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:43.249350 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:43.249355 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.250726 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.250729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:43.251835 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:43.251844 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.252852 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.252855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:43.252952 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:43.252971 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 14:29:43.253043 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 14:29:43.253431 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"mutex_wait_us":47,"thread_start_us":113,"threads_started":1}
I20260808 14:29:43.253572 18923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:43.247913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40626 (local address 127.7.125.66:33397)
0808 14:29:43.248388 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:43.248400 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:43.248424 (+ 24us) server_negotiation.cc:407] Connection header received
0808 14:29:43.248544 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:43.248550 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:43.248747 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:43.248915 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:43.249286 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.250104 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.251687 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.252490 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.252610 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:43.252757 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:43.253345 (+ 588us) server_negotiation.cc:299] Negotiation successful
0808 14:29:43.253445 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":110,"threads_started":1}
I20260808 14:29:43.254077 18918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:43.247140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:35507 (local address 127.0.0.1:33940)
0808 14:29:43.247931 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:43.247986 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:43.248121 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:43.248921 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:43.248927 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:43.248966 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:43.249162 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:43.249174 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.250630 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.250633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:43.252118 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:43.252128 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.253671 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.253677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:43.253787 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:43.253809 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 14:29:43.253872 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 14:29:43.253941 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"mutex_wait_us":123,"spinlock_wait_cycles":13440,"thread_start_us":83,"threads_started":1}
I20260808 14:29:43.254637 18920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:43.247240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33940 (local address 127.7.125.67:35507)
0808 14:29:43.247901 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:43.247914 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:43.248256 (+ 342us) server_negotiation.cc:407] Connection header received
0808 14:29:43.248341 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:43.248349 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:43.248568 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:43.248721 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:43.249297 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.250487 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.252242 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.253039 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.253136 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:43.254353 (+ 1217us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:43.254400 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 14:29:43.254489 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":117,"threads_started":1}
I20260808 14:29:43.256400 18695 tablet_service.cc:1467] Processing CreateTablet for tablet 8c79791a67cd43738a8b855d86f95f6d (DEFAULT_TABLE table=test-table [id=165914d485ac453da36cacb46268c23a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:43.256821 18837 tablet_service.cc:1467] Processing CreateTablet for tablet 8c79791a67cd43738a8b855d86f95f6d (DEFAULT_TABLE table=test-table [id=165914d485ac453da36cacb46268c23a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:43.257434 18695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c79791a67cd43738a8b855d86f95f6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:43.257755 18837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c79791a67cd43738a8b855d86f95f6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:43.264281 18925 tablet_bootstrap.cc:492] T 8c79791a67cd43738a8b855d86f95f6d P 9eff95b97a89456da07b4786c2eafd1d: Bootstrap starting.
I20260808 14:29:43.264719 18926 tablet_bootstrap.cc:492] T 8c79791a67cd43738a8b855d86f95f6d P f1f68b508deb4840aa5f41defb38f628: Bootstrap starting.
I20260808 14:29:43.265261 18551 tablet_service.cc:1467] Processing CreateTablet for tablet 8c79791a67cd43738a8b855d86f95f6d (DEFAULT_TABLE table=test-table [id=165914d485ac453da36cacb46268c23a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:29:43.266345 18551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c79791a67cd43738a8b855d86f95f6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:29:43.266633 18925 tablet_bootstrap.cc:654] T 8c79791a67cd43738a8b855d86f95f6d P 9eff95b97a89456da07b4786c2eafd1d: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:43.267298 18926 tablet_bootstrap.cc:654] T 8c79791a67cd43738a8b855d86f95f6d P f1f68b508deb4840aa5f41defb38f628: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:43.267552 18925 log.cc:824] T 8c79791a67cd43738a8b855d86f95f6d P 9eff95b97a89456da07b4786c2eafd1d: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:43.268119 18926 log.cc:824] T 8c79791a67cd43738a8b855d86f95f6d P f1f68b508deb4840aa5f41defb38f628: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:43.269677 18925 tablet_bootstrap.cc:492] T 8c79791a67cd43738a8b855d86f95f6d P 9eff95b97a89456da07b4786c2eafd1d: No bootstrap required, opened a new log
I20260808 14:29:43.269898 18925 ts_tablet_manager.cc:1397] T 8c79791a67cd43738a8b855d86f95f6d P 9eff95b97a89456da07b4786c2eafd1d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:29:43.270334 18926 tablet_bootstrap.cc:492] T 8c79791a67cd43738a8b855d86f95f6d P f1f68b508deb4840aa5f41defb38f628: No bootstrap required, opened a new log
I20260808 14:29:43.270529 18926 ts_tablet_manager.cc:1397] T 8c79791a67cd43738a8b855d86f95f6d P f1f68b508deb4840aa5f41defb38f628: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:29:43.272938 18925 raft_consensus.cc:348] T 8c79791a67cd43738a8b855d86f95f6d P 9eff95b97a89456da07b4786c2eafd1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eff95b97a89456da07b4786c2eafd1d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 35507 } } peers { permanent_uuid: "4e5db532e1024c2886c56fca7f74af59" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34713 } } peers { permanent_uuid: "f1f68b508deb4840aa5f41defb38f628" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 33397 } }
I20260808 14:29:43.273175 18925 raft_consensus.cc:374] T 8c79791a67cd43738a8b855d86f95f6d P 9eff95b97a89456da07b4786c2eafd1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:43.273237 18925 raft_consensus.cc:729] T 8c79791a67cd43738a8b855d86f95f6d P 9eff95b97a89456da07b4786c2eafd1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9eff95b97a89456da07b4786c2eafd1d, State: Initialized, Role: FOLLOWER
I20260808 14:29:43.273697 18925 consensus_queue.cc:260] T 8c79791a67cd43738a8b855d86f95f6d P 9eff95b97a89456da07b4786c2eafd1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eff95b97a89456da07b4786c2eafd1d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 35507 } } peers { permanent_uuid: "4e5db532e1024c2886c56fca7f74af59" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34713 } } peers { permanent_uuid: "f1f68b508deb4840aa5f41defb38f628" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 33397 } }
I20260808 14:29:43.273806 18929 tablet_bootstrap.cc:492] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59: Bootstrap starting.
I20260808 14:29:43.274390 18926 raft_consensus.cc:348] T 8c79791a67cd43738a8b855d86f95f6d P f1f68b508deb4840aa5f41defb38f628 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eff95b97a89456da07b4786c2eafd1d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 35507 } } peers { permanent_uuid: "4e5db532e1024c2886c56fca7f74af59" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34713 } } peers { permanent_uuid: "f1f68b508deb4840aa5f41defb38f628" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 33397 } }
I20260808 14:29:43.274650 18926 raft_consensus.cc:374] T 8c79791a67cd43738a8b855d86f95f6d P f1f68b508deb4840aa5f41defb38f628 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:43.274721 18926 raft_consensus.cc:729] T 8c79791a67cd43738a8b855d86f95f6d P f1f68b508deb4840aa5f41defb38f628 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1f68b508deb4840aa5f41defb38f628, State: Initialized, Role: FOLLOWER
I20260808 14:29:43.274765 18925 ts_tablet_manager.cc:1428] T 8c79791a67cd43738a8b855d86f95f6d P 9eff95b97a89456da07b4786c2eafd1d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 14:29:43.275079 18907 heartbeater.cc:503] Master 127.7.125.126:34199 was elected leader, sending a full tablet report...
I20260808 14:29:43.275210 18926 consensus_queue.cc:260] T 8c79791a67cd43738a8b855d86f95f6d P f1f68b508deb4840aa5f41defb38f628 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eff95b97a89456da07b4786c2eafd1d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 35507 } } peers { permanent_uuid: "4e5db532e1024c2886c56fca7f74af59" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34713 } } peers { permanent_uuid: "f1f68b508deb4840aa5f41defb38f628" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 33397 } }
I20260808 14:29:43.275888 18768 heartbeater.cc:503] Master 127.7.125.126:34199 was elected leader, sending a full tablet report...
I20260808 14:29:43.276494 18929 tablet_bootstrap.cc:654] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59: Neither blocks nor log segments found. Creating new log.
I20260808 14:29:43.276940 18926 ts_tablet_manager.cc:1428] T 8c79791a67cd43738a8b855d86f95f6d P f1f68b508deb4840aa5f41defb38f628: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 14:29:43.277309 18929 log.cc:824] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59: Log is configured to *not* fsync() on all Append() calls
I20260808 14:29:43.279521 18929 tablet_bootstrap.cc:492] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59: No bootstrap required, opened a new log
I20260808 14:29:43.279716 18929 ts_tablet_manager.cc:1397] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260808 14:29:43.283342 18929 raft_consensus.cc:348] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eff95b97a89456da07b4786c2eafd1d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 35507 } } peers { permanent_uuid: "4e5db532e1024c2886c56fca7f74af59" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34713 } } peers { permanent_uuid: "f1f68b508deb4840aa5f41defb38f628" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 33397 } }
I20260808 14:29:43.283573 18929 raft_consensus.cc:374] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:29:43.283614 18929 raft_consensus.cc:729] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e5db532e1024c2886c56fca7f74af59, State: Initialized, Role: FOLLOWER
I20260808 14:29:43.283993 18929 consensus_queue.cc:260] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eff95b97a89456da07b4786c2eafd1d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 35507 } } peers { permanent_uuid: "4e5db532e1024c2886c56fca7f74af59" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34713 } } peers { permanent_uuid: "f1f68b508deb4840aa5f41defb38f628" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 33397 } }
I20260808 14:29:43.284691 18622 heartbeater.cc:503] Master 127.7.125.126:34199 was elected leader, sending a full tablet report...
I20260808 14:29:43.285072 18929 ts_tablet_manager.cc:1428] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 14:29:43.398562 18933 raft_consensus.cc:482] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:29:43.398766 18933 raft_consensus.cc:504] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eff95b97a89456da07b4786c2eafd1d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 35507 } } peers { permanent_uuid: "4e5db532e1024c2886c56fca7f74af59" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34713 } } peers { permanent_uuid: "f1f68b508deb4840aa5f41defb38f628" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 33397 } }
I20260808 14:29:43.399935 18933 leader_election.cc:288] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9eff95b97a89456da07b4786c2eafd1d (127.7.125.67:35507), f1f68b508deb4840aa5f41defb38f628 (127.7.125.66:33397)
I20260808 14:29:43.405637 18923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:43.400317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:56521 (local address 127.7.125.66:33397)
0808 14:29:43.400459 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:43.400464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:43.400817 (+ 353us) server_negotiation.cc:407] Connection header received
0808 14:29:43.400921 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:43.400924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:43.400982 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:43.401071 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:43.401494 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.402280 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.404203 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.404932 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.405067 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:43.405123 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:43.405413 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 14:29:43.405492 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 14:29:43.405637 18920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:43.400291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:56179 (local address 127.7.125.67:35507)
0808 14:29:43.400420 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:43.400425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:43.400821 (+ 396us) server_negotiation.cc:407] Connection header received
0808 14:29:43.400921 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:43.400924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:43.400978 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:43.401068 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:29:43.401494 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.402316 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.404202 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.404916 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.405067 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:43.405123 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:43.405413 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 14:29:43.405492 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:29:43.405674 18934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:43.400199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:33397 (local address 127.7.125.65:56521)
0808 14:29:43.400657 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:43.400673 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:43.400818 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:43.401106 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:43.401109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:43.401138 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:43.401358 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:43.401365 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.402422 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.402425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:43.404038 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:43.404051 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.405068 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.405072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:43.405229 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:43.405285 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 14:29:43.405372 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 14:29:43.405466 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":179,"spinlock_wait_cycles":22784,"thread_start_us":132,"threads_started":1}
I20260808 14:29:43.405674 18935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:43.400260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:35507 (local address 127.7.125.65:56179)
0808 14:29:43.400700 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:43.400713 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:29:43.400818 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:29:43.401106 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:29:43.401109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:29:43.401137 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:29:43.401358 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:43.401365 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.402426 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.402428 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:43.404038 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:29:43.404051 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.405068 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.405072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:29:43.405232 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:43.405292 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 14:29:43.405370 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 14:29:43.405465 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":346,"spinlock_wait_cycles":27648,"thread_start_us":112,"threads_started":1}
I20260808 14:29:43.406425 18857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c79791a67cd43738a8b855d86f95f6d" candidate_uuid: "4e5db532e1024c2886c56fca7f74af59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9eff95b97a89456da07b4786c2eafd1d" is_pre_election: true
I20260808 14:29:43.406412 18717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c79791a67cd43738a8b855d86f95f6d" candidate_uuid: "4e5db532e1024c2886c56fca7f74af59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1f68b508deb4840aa5f41defb38f628" is_pre_election: true
I20260808 14:29:43.406757 18857 raft_consensus.cc:2454] T 8c79791a67cd43738a8b855d86f95f6d P 9eff95b97a89456da07b4786c2eafd1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e5db532e1024c2886c56fca7f74af59 in term 0.
I20260808 14:29:43.406757 18717 raft_consensus.cc:2454] T 8c79791a67cd43738a8b855d86f95f6d P f1f68b508deb4840aa5f41defb38f628 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e5db532e1024c2886c56fca7f74af59 in term 0.
I20260808 14:29:43.407327 18502 leader_election.cc:302] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4e5db532e1024c2886c56fca7f74af59, 9eff95b97a89456da07b4786c2eafd1d, f1f68b508deb4840aa5f41defb38f628; no voters:
I20260808 14:29:43.407648 18933 raft_consensus.cc:2781] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:29:43.407739 18933 raft_consensus.cc:482] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:29:43.407784 18933 raft_consensus.cc:3037] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:43.408836 18933 raft_consensus.cc:504] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eff95b97a89456da07b4786c2eafd1d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 35507 } } peers { permanent_uuid: "4e5db532e1024c2886c56fca7f74af59" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34713 } } peers { permanent_uuid: "f1f68b508deb4840aa5f41defb38f628" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 33397 } }
I20260808 14:29:43.409214 18933 leader_election.cc:288] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59 [CANDIDATE]: Term 1 election: Requested vote from peers 9eff95b97a89456da07b4786c2eafd1d (127.7.125.67:35507), f1f68b508deb4840aa5f41defb38f628 (127.7.125.66:33397)
I20260808 14:29:43.409575 18857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c79791a67cd43738a8b855d86f95f6d" candidate_uuid: "4e5db532e1024c2886c56fca7f74af59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9eff95b97a89456da07b4786c2eafd1d"
I20260808 14:29:43.409698 18857 raft_consensus.cc:3037] T 8c79791a67cd43738a8b855d86f95f6d P 9eff95b97a89456da07b4786c2eafd1d [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:43.409704 18717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c79791a67cd43738a8b855d86f95f6d" candidate_uuid: "4e5db532e1024c2886c56fca7f74af59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1f68b508deb4840aa5f41defb38f628"
I20260808 14:29:43.409868 18717 raft_consensus.cc:3037] T 8c79791a67cd43738a8b855d86f95f6d P f1f68b508deb4840aa5f41defb38f628 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:29:43.410765 18857 raft_consensus.cc:2454] T 8c79791a67cd43738a8b855d86f95f6d P 9eff95b97a89456da07b4786c2eafd1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e5db532e1024c2886c56fca7f74af59 in term 1.
I20260808 14:29:43.411160 18502 leader_election.cc:302] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e5db532e1024c2886c56fca7f74af59, 9eff95b97a89456da07b4786c2eafd1d; no voters:
I20260808 14:29:43.411250 18717 raft_consensus.cc:2454] T 8c79791a67cd43738a8b855d86f95f6d P f1f68b508deb4840aa5f41defb38f628 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e5db532e1024c2886c56fca7f74af59 in term 1.
I20260808 14:29:43.411334 18933 raft_consensus.cc:2781] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:29:43.411574 18933 raft_consensus.cc:686] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59 [term 1 LEADER]: Becoming Leader. State: Replica: 4e5db532e1024c2886c56fca7f74af59, State: Running, Role: LEADER
I20260808 14:29:43.411924 18933 consensus_queue.cc:237] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eff95b97a89456da07b4786c2eafd1d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 35507 } } peers { permanent_uuid: "4e5db532e1024c2886c56fca7f74af59" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34713 } } peers { permanent_uuid: "f1f68b508deb4840aa5f41defb38f628" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 33397 } }
I20260808 14:29:43.415143 18419 catalog_manager.cc:5495] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e5db532e1024c2886c56fca7f74af59 (127.7.125.65). New cstate: current_term: 1 leader_uuid: "4e5db532e1024c2886c56fca7f74af59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eff95b97a89456da07b4786c2eafd1d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 35507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e5db532e1024c2886c56fca7f74af59" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1f68b508deb4840aa5f41defb38f628" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 33397 } health_report { overall_health: UNKNOWN } } }
I20260808 14:29:43.443046 18922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:43.439367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45116 (local address 127.7.125.65:34713)
0808 14:29:43.439514 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 14:29:43.439519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:29:43.439535 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:29:43.439633 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:29:43.439636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:29:43.439691 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:29:43.439801 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:29:43.440079 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.440588 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.441515 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:29:43.441731 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:29:43.441842 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:29:43.442185 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:29:43.442588 (+ 403us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:29:43.442726 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 14:29:43.442776 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 14:29:43.442832 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 14:29:43.446653 18908 tablet.cc:2369] T 8c79791a67cd43738a8b855d86f95f6d P 9eff95b97a89456da07b4786c2eafd1d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:29:43.449306 18857 raft_consensus.cc:1261] T 8c79791a67cd43738a8b855d86f95f6d P 9eff95b97a89456da07b4786c2eafd1d [term 1 FOLLOWER]: Refusing update from remote peer 4e5db532e1024c2886c56fca7f74af59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:29:43.449333 18717 raft_consensus.cc:1261] T 8c79791a67cd43738a8b855d86f95f6d P f1f68b508deb4840aa5f41defb38f628 [term 1 FOLLOWER]: Refusing update from remote peer 4e5db532e1024c2886c56fca7f74af59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:29:43.450016 18936 consensus_queue.cc:1035] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9eff95b97a89456da07b4786c2eafd1d" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 35507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:29:43.450212 18933 consensus_queue.cc:1035] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1f68b508deb4840aa5f41defb38f628" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 33397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:29:43.466518 18419 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:29:43.466693 18419 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:29:43.468255 18941 mvcc.cc:205] Tried to move back new op lower bound from 7316272674601259008 to 7316272674457874432. Current Snapshot: MvccSnapshot[applied={T|T < 7316272674601259008}]
I20260808 14:29:43.469455 18419 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b538c01084014aa1a6633e1f7faf48ac: Sending DeleteTablet for 3 replicas of tablet 8c79791a67cd43738a8b855d86f95f6d
I20260808 14:29:43.470175 18837 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c79791a67cd43738a8b855d86f95f6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:43 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:33940
I20260808 14:29:43.470993 18551 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c79791a67cd43738a8b855d86f95f6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:43 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:45106
I20260808 14:29:43.471050 18695 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c79791a67cd43738a8b855d86f95f6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:29:43 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:40626
I20260808 14:29:43.471463 18949 tablet_replica.cc:330] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59: stopping tablet replica
I20260808 14:29:43.471805 18949 raft_consensus.cc:2229] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:29:43.472154 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 18485
I20260808 14:29:43.472262 18949 raft_consensus.cc:2258] T 8c79791a67cd43738a8b855d86f95f6d P 4e5db532e1024c2886c56fca7f74af59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:43.474825 18948 tablet_replica.cc:330] T 8c79791a67cd43738a8b855d86f95f6d P 9eff95b97a89456da07b4786c2eafd1d: stopping tablet replica
I20260808 14:29:43.475181 18948 raft_consensus.cc:2229] T 8c79791a67cd43738a8b855d86f95f6d P 9eff95b97a89456da07b4786c2eafd1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:29:43.475500 18948 raft_consensus.cc:2258] T 8c79791a67cd43738a8b855d86f95f6d P 9eff95b97a89456da07b4786c2eafd1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:43.475930 18950 tablet_replica.cc:330] T 8c79791a67cd43738a8b855d86f95f6d P f1f68b508deb4840aa5f41defb38f628: stopping tablet replica
I20260808 14:29:43.476612 18950 raft_consensus.cc:2229] T 8c79791a67cd43738a8b855d86f95f6d P f1f68b508deb4840aa5f41defb38f628 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:29:43.476922 18950 raft_consensus.cc:2258] T 8c79791a67cd43738a8b855d86f95f6d P f1f68b508deb4840aa5f41defb38f628 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:29:43.477160 18948 ts_tablet_manager.cc:1905] T 8c79791a67cd43738a8b855d86f95f6d P 9eff95b97a89456da07b4786c2eafd1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:29:43.478336 18950 ts_tablet_manager.cc:1905] T 8c79791a67cd43738a8b855d86f95f6d P f1f68b508deb4840aa5f41defb38f628: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:29:43.480100 18948 ts_tablet_manager.cc:1918] T 8c79791a67cd43738a8b855d86f95f6d P 9eff95b97a89456da07b4786c2eafd1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:29:43.480216 18948 log.cc:1196] T 8c79791a67cd43738a8b855d86f95f6d P 9eff95b97a89456da07b4786c2eafd1d: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/8c79791a67cd43738a8b855d86f95f6d
I20260808 14:29:43.480600 18948 ts_tablet_manager.cc:1939] T 8c79791a67cd43738a8b855d86f95f6d P 9eff95b97a89456da07b4786c2eafd1d: Deleting consensus metadata
I20260808 14:29:43.481355 18950 ts_tablet_manager.cc:1918] T 8c79791a67cd43738a8b855d86f95f6d P f1f68b508deb4840aa5f41defb38f628: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:29:43.481451 18950 log.cc:1196] T 8c79791a67cd43738a8b855d86f95f6d P f1f68b508deb4840aa5f41defb38f628: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/8c79791a67cd43738a8b855d86f95f6d
I20260808 14:29:43.481762 18950 ts_tablet_manager.cc:1939] T 8c79791a67cd43738a8b855d86f95f6d P f1f68b508deb4840aa5f41defb38f628: Deleting consensus metadata
W20260808 14:29:43.481905 18405 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.125.65:34713 (error 108)
I20260808 14:29:43.482133 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 18627
I20260808 14:29:43.482249 18403 catalog_manager.cc:4841] TS 9eff95b97a89456da07b4786c2eafd1d (127.7.125.67:35507): tablet 8c79791a67cd43738a8b855d86f95f6d (table test-table [id=165914d485ac453da36cacb46268c23a]) successfully deleted
I20260808 14:29:43.484640 18918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:43.484034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:34713 (local address 127.0.0.1:45128)
0808 14:29:43.484504 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:43.484564 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.65:34713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391}
W20260808 14:29:43.484903 18405 catalog_manager.cc:4584] TS 4e5db532e1024c2886c56fca7f74af59 (127.7.125.65:34713): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8c79791a67cd43738a8b855d86f95f6d: Network error: Client connection negotiation failed: client connection to 127.7.125.65:34713: connect: Connection refused (error 111)
I20260808 14:29:43.489544 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 18773
I20260808 14:29:43.491912 18918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:29:43.489688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:33397 (local address 127.0.0.1:40634)
0808 14:29:43.491728 (+ 2040us) negotiation.cc:107] Waiting for socket to connect
0808 14:29:43.491804 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.66:33397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1922}
I20260808 14:29:43.496728 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 18386
2026-08-08T14:29:43Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2433 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:29:43 dist-test-slave-w206 krb5kdc[18954](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:29:43 dist-test-slave-w206 krb5kdc[18954](info): set up 2 sockets
Aug 08 14:29:43 dist-test-slave-w206 krb5kdc[18954](info): commencing operation
krb5kdc: starting...
W20260808 14:29:44.687408 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:29:44 dist-test-slave-w206 krb5kdc[18954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:29:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:29:44Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.125.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.125.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.125.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/rangeradmin_127.7.125.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.125.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/rangeradmin_127.7.125.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.125.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.125.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.125.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/rangerlookup_127.7.125.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.125.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/rangerlookup_127.7.125.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.125.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.125.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.125.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/HTTP_127.7.125.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.125.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/HTTP_127.7.125.84@KRBTEST.COM.keytab.
I20260808 14:29:44.873736 7669 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskngm_Z6/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/postgres -l logfile start
2026-08-08 14:29:47.450 UTC [18984] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 14:29:47.450 UTC [18984] LOG: listening on IPv4 address "127.7.125.84", port 36355
2026-08-08 14:29:47.453 UTC [18984] LOG: listening on Unix socket "/tmp/.s.PGSQL.36355"
2026-08-08 14:29:47.476 UTC [18987] LOG: database system was shut down at 2026-08-08 14:29:46 UTC
2026-08-08 14:29:47.480 UTC [18984] LOG: database system is ready to accept connections
I20260808 14:29:48.467861 7669 mini_postgres.cc:96] Postgres bound to 36355
2026-08-08 14:29:48.476 UTC [18995] FATAL: database "slave" does not exist
127.7.125.84:36355 - accepting connections
I20260808 14:29:48.477229 7669 mini_ranger.cc:161] Starting Ranger...
I20260808 14:29:48.493829 7669 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 14:29:48.999584 7669 mini_ranger.cc:87] Created ranger Postgres database
I20260808 14:29:48.999697 7669 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ranger-admin
2026-08-08 14:29:49,334 [I] DB FLAVOR :POSTGRES
2026-08-08 14:29:49,334 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 14:29:49,334 [I] Checking connection..
2026-08-08 14:29:49,334 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:36355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 14:29:49,605 [I] Checking connection passed.
2026-08-08 14:29:49,605 [I] --------- Verifying version history table ---------
2026-08-08 14:29:49,605 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:36355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 14:29:49,856 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 14:29:49,856 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:36355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 14:29:50,121 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 14:29:50,121 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 14:29:50,122 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:36355/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 14:29:50.375 UTC [19061] WARNING: there is no transaction in progress
2026-08-08 14:29:50,383 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 14:29:50,383 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:36355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 14:29:50,627 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 14:29:50,627 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 14:29:50,627 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:36355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 14:29:50,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:36355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 14:29:51,174 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:36355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-w206.c.gcp-upstream.internal','N') ;"
2026-08-08 14:29:51,433 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:36355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 14:29:51,701 [I] Table x_portal_user does not exist in database ranger
2026-08-08 14:29:51,702 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:36355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 14:29:51,965 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 14:29:51,965 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:36355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 14:29:52,212 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:36355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 14:29:52,485 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 14:29:52,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:36355/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 14:29:53.531 UTC [19172] WARNING: there is no transaction in progress
2026-08-08 14:29:53.549 UTC [19172] WARNING: there is no transaction in progress
2026-08-08 14:29:53.568 UTC [19172] WARNING: there is no transaction in progress
2026-08-08 14:29:53.586 UTC [19172] WARNING: there is no transaction in progress
2026-08-08 14:29:53.604 UTC [19172] WARNING: there is no transaction in progress
2026-08-08 14:29:53.624 UTC [19172] WARNING: there is no transaction in progress
2026-08-08 14:29:53.682 UTC [19172] WARNING: there is no transaction in progress
2026-08-08 14:29:53.690 UTC [19172] WARNING: there is no transaction in progress
2026-08-08 14:29:53.706 UTC [19172] WARNING: there is no transaction in progress
2026-08-08 14:29:53.717 UTC [19172] WARNING: there is no transaction in progress
2026-08-08 14:29:53.728 UTC [19172] WARNING: there is no transaction in progress
2026-08-08 14:29:53.744 UTC [19172] WARNING: there is no transaction in progress
2026-08-08 14:29:53.754 UTC [19172] WARNING: there is no transaction in progress
2026-08-08 14:29:53.761 UTC [19172] WARNING: there is no transaction in progress
2026-08-08 14:29:53.769 UTC [19172] WARNING: there is no transaction in progress
2026-08-08 14:29:53.777 UTC [19172] WARNING: there is no transaction in progress
2026-08-08 14:29:54,480 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 14:29:54,480 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:36355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 14:29:54,710 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 14:29:54,711 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:36355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 14:29:54,963 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 14:29:54,963 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:36355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 14:29:55,228 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 14:29:55,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:36355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 14:29:55,495 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:36355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 14:29:55,770 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:36355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-w206.c.gcp-upstream.internal';"
2026-08-08 14:29:56,030 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 14:29:56,031 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:36355/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 14:29:56,275 [I] DB_PATCHES have already been applied
I20260808 14:29:56.281194 7669 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ranger-admin:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskngm_Z6/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 14:29:56.281272 7669 mini_ranger.cc:193] Using host: 127.7.125.84
I20260808 14:29:56.285864 7669 mini_ranger.cc:227] Ranger admin URL: http://127.7.125.84:44163
Aug 08, 2026 2:29:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:29:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:29:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 2:29:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 2:29:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 2:29:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 2:29:57 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 14:29:57.462033 7669 mini_ranger.cc:160] Time spent starting Ranger: real 8.985s user 0.000s sys 0.013s
Aug 08, 2026 2:30:18 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 2:30:18 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 08, 2026 2:30:18 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 2:30:18 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 2:30:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 2:30:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 2:30:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 2:30:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 2:30:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 2:30:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 2:30:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 2:30:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 2:30:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 2:30:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 2:30:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 2:30:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 2:30:18 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 2:30:19 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 14:30:20.215608 7669 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.125.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.125.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.125.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/rangerkms_127.7.125.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.125.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/rangerkms_127.7.125.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.125.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.125.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.125.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/HTTP_127.7.125.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.125.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/HTTP_127.7.125.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 14:30:20 dist-test-slave-w206 krb5kdc[18954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199420, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 14:30:20.336020 7669 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 14:30:20.347896 7669 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 14:30:21.311865 7669 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 14:30:21.311976 7669 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ranger-kms
I20260808 14:30:21.312088 7669 mini_ranger_kms.cc:227] Using postgres at 127.7.125.84:36355
2026-08-08 14:30:21,545 [I] DB FLAVOR :POSTGRES
2026-08-08 14:30:21,546 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 14:30:21,546 [I] Checking connection
2026-08-08 14:30:21,546 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:36355/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 14:30:21,806 [I] connection success
2026-08-08 14:30:21,806 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 14:30:21,806 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 14:30:21,807 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:36355/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 14:30:22,085 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 14:30:22,085 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 14:30:22,085 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 14:30:22,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:36355/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 14:30:22,372 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 14:30:22.878887 7669 mini_ranger_kms.cc:310] Created kms service
I20260808 14:30:23.089960 7669 mini_ranger_kms.cc:326] Created kudu user
I20260808 14:30:23.137796 7669 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 14:30:23 dist-test-slave-w206 krb5kdc[18954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199420, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.125.84@KRBTEST.COM
I20260808 14:30:23.588644 7669 mini_ranger_kms.cc:384] Added ranger policy
I20260808 14:30:23.588824 7669 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ranger-kms:/tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 14:30:23.588850 7669 mini_ranger_kms.cc:240] Using host: 127.7.125.84
I20260808 14:30:23.594627 7669 mini_ranger_kms.cc:276] Ranger KMS PID: 19363
I20260808 14:30:23.594832 7669 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.125.84:53237
Aug 08, 2026 2:30:24 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:30:24 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:30:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 2:30:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 2:30:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 2:30:25 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 14:30:25 dist-test-slave-w206 krb5kdc[18954](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786199425, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.125.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:30:25.970876 7669 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.635s user 0.014s sys 0.206s
I20260808 14:30:25.971062 7669 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 14:30:25.971197 7669 mini_ranger_kms.cc:401] 127.7.125.84:53237/kms/v1/keys
Aug 08 14:30:26 dist-test-slave-w206 krb5kdc[18954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199425, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.125.84@KRBTEST.COM for HTTP/127.7.125.84@KRBTEST.COM
Aug 08 14:30:26 dist-test-slave-w206 krb5kdc[18954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199425, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.125.84@KRBTEST.COM for HTTP/127.7.125.84@KRBTEST.COM
Aug 08 14:30:28 dist-test-slave-w206 krb5kdc[18954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:30:28.866420 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:33027
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:40137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:33027
--ranger_config_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.125.84:53237/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:30:28.984509 19418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:30:28.984829 19418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:30:28.984875 19418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:30:28.988942 19418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:30:28.989003 19418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:30:28.989022 19418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:30:28.989038 19418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:30:28.989053 19418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:30:28.994699 19418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:40137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.125.84:53237/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:33027
--ranger_config_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:33027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:30:28.995889 19418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:30:28.996956 19418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:30:29.006101 19428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:30:29.006079 19426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:30:29.006134 19424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:30:29.006649 19418 server_base.cc:1034] running on GCE node
I20260808 14:30:29.007296 19418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:30:29.008507 19418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:30:29.009691 19418 hybrid_clock.cc:648] HybridClock initialized: now 1786199429009654 us; error 55 us; skew 500 ppm
Aug 08 14:30:29 dist-test-slave-w206 krb5kdc[18954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199429, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:30:29.016810 19418 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:30:29.017915 19418 webserver.cc:460] Webserver started at http://127.7.125.126:42527/ using document root <none> and password file <none>
I20260808 14:30:29.018494 19418 fs_manager.cc:359] Metadata directory not provided
I20260808 14:30:29.018550 19418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:30:29.018800 19418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 14:30:29 dist-test-slave-w206 krb5kdc[18954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199429, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for HTTP/127.7.125.84@KRBTEST.COM
I20260808 14:30:29.067979 19418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "5734b27b820240babf70d56cfdf18428"
format_stamp: "Formatted at 2026-08-08 14:30:29 on dist-test-slave-w206"
server_key: "7e3f89d750c74be2cc305436b8a39920"
server_key_iv: "ecda0a77f82bda7ad5aeee1110253ef5"
server_key_version: "kuduclusterkey@0"
I20260808 14:30:29.068612 19418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "5734b27b820240babf70d56cfdf18428"
format_stamp: "Formatted at 2026-08-08 14:30:29 on dist-test-slave-w206"
server_key: "7e3f89d750c74be2cc305436b8a39920"
server_key_iv: "ecda0a77f82bda7ad5aeee1110253ef5"
server_key_version: "kuduclusterkey@0"
I20260808 14:30:29.072851 19418 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:30:29.088248 19435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:30:29.089711 19418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:30:29.089860 19418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "5734b27b820240babf70d56cfdf18428"
format_stamp: "Formatted at 2026-08-08 14:30:29 on dist-test-slave-w206"
server_key: "7e3f89d750c74be2cc305436b8a39920"
server_key_iv: "ecda0a77f82bda7ad5aeee1110253ef5"
server_key_version: "kuduclusterkey@0"
I20260808 14:30:29.090003 19418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:30:29.102150 19418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:30:29.105218 19418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:30:29.105444 19418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:30:29.113755 19418 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:33027
I20260808 14:30:29.113816 19490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:33027 every 8 connection(s)
I20260808 14:30:29.114861 19418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:30:29.115646 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 19418
I20260808 14:30:29.115800 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:30:29.116103 7669 external_mini_cluster.cc:1428] Setting key 5415a3fd7aed61c8e61a7e1c9289b30a
I20260808 14:30:29.118376 19491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 14:30:29 dist-test-slave-w206 krb5kdc[18954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:30:29.125391 19491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428: Bootstrap starting.
I20260808 14:30:29.128077 19491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428: Neither blocks nor log segments found. Creating new log.
I20260808 14:30:29.129458 19491 log.cc:824] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428: Log is configured to *not* fsync() on all Append() calls
I20260808 14:30:29.130743 19494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:30:29.117660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49144 (local address 127.7.125.126:33027)
0808 14:30:29.118091 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0808 14:30:29.118110 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 14:30:29.118157 (+ 47us) server_negotiation.cc:407] Connection header received
0808 14:30:29.118870 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:30:29.118888 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:30:29.119303 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:30:29.119644 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:30:29.120533 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:30:29.121533 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:30:29.122411 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:30:29.122887 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:30:29.123102 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:30:29.125752 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:30:29.125758 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:30:29.125772 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:30:29.125800 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:30:29.127613 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:30:29.128098 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:30:29.128104 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:30:29.128108 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:30:29.128273 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:30:29.128576 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:30:29.128580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:30:29.128582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:30:29.128995 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:30:29.129209 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 14:30:29.129536 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 14:30:29.129850 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":155,"threads_started":1}
I20260808 14:30:29.131997 19491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428: No bootstrap required, opened a new log
I20260808 14:30:29.136264 19491 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5734b27b820240babf70d56cfdf18428" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33027 } }
I20260808 14:30:29.136560 19491 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:30:29.136617 19491 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5734b27b820240babf70d56cfdf18428, State: Initialized, Role: FOLLOWER
I20260808 14:30:29.137094 19491 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5734b27b820240babf70d56cfdf18428" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33027 } }
I20260808 14:30:29.137252 19491 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:30:29.137351 19491 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:30:29.137452 19491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:30:29.138650 19491 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5734b27b820240babf70d56cfdf18428" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33027 } }
I20260808 14:30:29.139068 19491 leader_election.cc:302] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5734b27b820240babf70d56cfdf18428; no voters:
I20260808 14:30:29.139423 19491 leader_election.cc:288] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:30:29.139573 19496 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:30:29.139828 19496 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428 [term 1 LEADER]: Becoming Leader. State: Replica: 5734b27b820240babf70d56cfdf18428, State: Running, Role: LEADER
I20260808 14:30:29.140218 19496 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5734b27b820240babf70d56cfdf18428" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33027 } }
I20260808 14:30:29.140826 19491 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:30:29.141849 19498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5734b27b820240babf70d56cfdf18428. Latest consensus state: current_term: 1 leader_uuid: "5734b27b820240babf70d56cfdf18428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5734b27b820240babf70d56cfdf18428" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33027 } } }
I20260808 14:30:29.141989 19498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428 [sys.catalog]: This master's current role is: LEADER
I20260808 14:30:29.142318 19497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5734b27b820240babf70d56cfdf18428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5734b27b820240babf70d56cfdf18428" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33027 } } }
I20260808 14:30:29.142395 19497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428 [sys.catalog]: This master's current role is: LEADER
I20260808 14:30:29.143275 19491 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 14:30:29.143285 19505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:30:29.150112 19505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:30:29.161257 19505 catalog_manager.cc:1347] Generated new cluster ID: 9882cf4951974f119f75b3a9fab72fd0
I20260808 14:30:29.161465 19505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:30:29.173730 19505 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:30:29.175390 19505 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:30:29.189744 19505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428: Generated new TSK 0
I20260808 14:30:29.190872 19505 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.125.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
I20260808 14:30:29.993026 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.125.84:53237/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:33027
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:40137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:30:30.133522 19537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:30:30.133826 19537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:30:30.133905 19537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:30:30.138379 19537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:30:30.138468 19537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:30:30.138563 19537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:30:30.144622 19537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:40137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.125.84:53237/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:33027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:30:30.145816 19537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:30:30.146867 19537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:30:30.157218 19542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:30:30.157272 19543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:30:30.157213 19545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:30:30.157802 19537 server_base.cc:1034] running on GCE node
I20260808 14:30:30.158267 19537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:30:30.158949 19537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:30:30.160204 19537 hybrid_clock.cc:648] HybridClock initialized: now 1786199430160183 us; error 41 us; skew 500 ppm
Aug 08 14:30:30 dist-test-slave-w206 krb5kdc[18954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199430, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:30:30.163358 19537 init.cc:377] Logged in from keytab as kudu/127.7.125.65@KRBTEST.COM (short username kudu)
I20260808 14:30:30.164407 19537 webserver.cc:460] Webserver started at http://127.7.125.65:42503/ using document root <none> and password file <none>
I20260808 14:30:30.165011 19537 fs_manager.cc:359] Metadata directory not provided
I20260808 14:30:30.165076 19537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:30:30.165302 19537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 14:30:30 dist-test-slave-w206 krb5kdc[18954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199430, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for HTTP/127.7.125.84@KRBTEST.COM
I20260808 14:30:30.184346 19537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "060b9bcbfce147b39d37453cef4ff3f0"
format_stamp: "Formatted at 2026-08-08 14:30:30 on dist-test-slave-w206"
server_key: "1ddb5c1514a0ff480ae8960cabbe8066"
server_key_iv: "73732754445eba592cc316bc81071d52"
server_key_version: "kuduclusterkey@0"
I20260808 14:30:30.185034 19537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "060b9bcbfce147b39d37453cef4ff3f0"
format_stamp: "Formatted at 2026-08-08 14:30:30 on dist-test-slave-w206"
server_key: "1ddb5c1514a0ff480ae8960cabbe8066"
server_key_iv: "73732754445eba592cc316bc81071d52"
server_key_version: "kuduclusterkey@0"
I20260808 14:30:30.189507 19537 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:30:30.210328 19554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:30:30.212265 19537 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260808 14:30:30.212468 19537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "060b9bcbfce147b39d37453cef4ff3f0"
format_stamp: "Formatted at 2026-08-08 14:30:30 on dist-test-slave-w206"
server_key: "1ddb5c1514a0ff480ae8960cabbe8066"
server_key_iv: "73732754445eba592cc316bc81071d52"
server_key_version: "kuduclusterkey@0"
I20260808 14:30:30.212625 19537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:30:30.235462 19537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:30:30.239825 19537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:30:30.240147 19537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:30:30.241015 19537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:30:30.242461 19537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:30:30.242585 19537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:30:30.242738 19537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:30:30.242827 19537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:30:30.257822 19537 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:37429
I20260808 14:30:30.259178 19537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:30:30.260740 19675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:37429 every 8 connection(s)
I20260808 14:30:30.263553 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 19537
I20260808 14:30:30.263695 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:30:30.264038 7669 external_mini_cluster.cc:1428] Setting key 37f1763f3e8ad56220c2bc268194aa4c
WARNING: no policy specified for kudu/127.7.125.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.66@KRBTEST.COM" created.
Aug 08 14:30:30 dist-test-slave-w206 krb5kdc[18954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199430, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:30:30.306100 19679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:30:30.262919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:45545 (local address 127.7.125.126:33027)
0808 14:30:30.270906 (+ 7987us) server_negotiation.cc:206] Beginning negotiation
0808 14:30:30.270913 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:30:30.270934 (+ 21us) server_negotiation.cc:407] Connection header received
0808 14:30:30.271008 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:30:30.271013 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:30:30.271083 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:30:30.271199 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:30:30.273185 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:30:30.274092 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:30:30.280604 (+ 6512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:30:30.281016 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:30:30.281204 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:30:30.298937 (+ 17733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:30:30.298943 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:30:30.298946 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:30:30.299009 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:30:30.303222 (+ 4213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:30:30.304015 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:30:30.304019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:30:30.304021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:30:30.304122 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:30:30.304624 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:30:30.304627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:30:30.304629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:30:30.304901 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:30:30.304981 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 14:30:30.305731 (+ 750us) server_negotiation.cc:299] Negotiation successful
0808 14:30:30.305888 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7858,"thread_start_us":115,"threads_started":1}
I20260808 14:30:30.308318 19678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:30:30.262184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:33027 (local address 127.7.125.65:45545)
0808 14:30:30.263977 (+ 1793us) negotiation.cc:107] Waiting for socket to connect
0808 14:30:30.264059 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0808 14:30:30.265411 (+ 1352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:30:30.271625 (+ 6214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:30:30.271643 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:30:30.272138 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:30:30.272969 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:30:30.272993 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:30:30.274266 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:30:30.274272 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:30:30.275964 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:30:30.275980 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:30:30.281295 (+ 5315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:30:30.281304 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:30:30.286779 (+ 5475us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:30:30.287813 (+ 1034us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:30:30.287841 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:30:30.297499 (+ 9658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:30:30.303408 (+ 5909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:30:30.303417 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:30:30.303439 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:30:30.303883 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:30:30.304244 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:30:30.304247 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:30:30.304249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:30:30.304498 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:30:30.305067 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:30:30.305075 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:30:30.305437 (+ 362us) client_negotiation.cc:769] Sending connection context
0808 14:30:30.306702 (+ 1265us) client_negotiation.cc:240] Negotiation successful
0808 14:30:30.307145 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1370,"thread_start_us":167,"threads_started":1}
I20260808 14:30:30.310259 19676 heartbeater.cc:347] Connected to a master server at 127.7.125.126:33027
I20260808 14:30:30.310564 19676 heartbeater.cc:464] Registering TS with master...
I20260808 14:30:30.311290 19676 heartbeater.cc:511] Master 127.7.125.126:33027 requested a full tablet report, sending...
I20260808 14:30:30.313220 19451 ts_manager.cc:194] Registered new tserver with Master: 060b9bcbfce147b39d37453cef4ff3f0 (127.7.125.65:37429)
I20260808 14:30:30.314963 19451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:45545
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
I20260808 14:30:30.322094 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.125.84:53237/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:33027
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:40137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 14:30:30 dist-test-slave-w206 krb5kdc[18954](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786199430, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 14:30:30.500905 19684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:30:30.501334 19684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:30:30.501435 19684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:30:30.507635 19684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:30:30.507746 19684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:30:30.507854 19684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:30:30.516664 19684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:40137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.125.84:53237/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:33027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:30:30.518275 19684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:30:30.520038 19684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:30:30.533987 19694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:30:30.533963 19692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:30:30.533959 19691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:30:30.534619 19684 server_base.cc:1034] running on GCE node
I20260808 14:30:30.535210 19684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:30:30.536209 19684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:30:30.537464 19684 hybrid_clock.cc:648] HybridClock initialized: now 1786199430537403 us; error 46 us; skew 500 ppm
Aug 08 14:30:30 dist-test-slave-w206 krb5kdc[18954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199430, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:30:30.541833 19684 init.cc:377] Logged in from keytab as kudu/127.7.125.66@KRBTEST.COM (short username kudu)
I20260808 14:30:30.543124 19684 webserver.cc:460] Webserver started at http://127.7.125.66:42593/ using document root <none> and password file <none>
I20260808 14:30:30.543744 19684 fs_manager.cc:359] Metadata directory not provided
I20260808 14:30:30.543864 19684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:30:30.544138 19684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 14:30:30 dist-test-slave-w206 krb5kdc[18954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199430, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for HTTP/127.7.125.84@KRBTEST.COM
I20260808 14:30:30.560959 19684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "eb429499d109402cb2afc9f05caf93b5"
format_stamp: "Formatted at 2026-08-08 14:30:30 on dist-test-slave-w206"
server_key: "3d4622fcd7f33ee53d881179ed42f6f4"
server_key_iv: "d094861e56cb9a779c8fe19ccf8646cd"
server_key_version: "kuduclusterkey@0"
I20260808 14:30:30.561746 19684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "eb429499d109402cb2afc9f05caf93b5"
format_stamp: "Formatted at 2026-08-08 14:30:30 on dist-test-slave-w206"
server_key: "3d4622fcd7f33ee53d881179ed42f6f4"
server_key_iv: "d094861e56cb9a779c8fe19ccf8646cd"
server_key_version: "kuduclusterkey@0"
I20260808 14:30:30.565959 19684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 14:30:30.598202 19701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:30:30.600239 19684 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:30:30.600414 19684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "eb429499d109402cb2afc9f05caf93b5"
format_stamp: "Formatted at 2026-08-08 14:30:30 on dist-test-slave-w206"
server_key: "3d4622fcd7f33ee53d881179ed42f6f4"
server_key_iv: "d094861e56cb9a779c8fe19ccf8646cd"
server_key_version: "kuduclusterkey@0"
I20260808 14:30:30.600574 19684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:30:30.632778 19684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:30:30.637321 19684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:30:30.637564 19684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:30:30.638430 19684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:30:30.639829 19684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:30:30.639909 19684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:30:30.639995 19684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:30:30.640029 19684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:30:30.653944 19684 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:33647
I20260808 14:30:30.656909 19818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:33647 every 8 connection(s)
I20260808 14:30:30.657366 19684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:30:30.660120 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 19684
I20260808 14:30:30.660262 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:30:30.660619 7669 external_mini_cluster.cc:1428] Setting key 176c08d6fdd914cf17a23b53c768dcde
Aug 08 14:30:30 dist-test-slave-w206 krb5kdc[18954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199430, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:30:30.677434 19679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:30:30.662161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:33783 (local address 127.7.125.126:33027)
0808 14:30:30.662288 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 14:30:30.662293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:30:30.662801 (+ 508us) server_negotiation.cc:407] Connection header received
0808 14:30:30.664137 (+ 1336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:30:30.664142 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:30:30.664196 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:30:30.664258 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:30:30.665973 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:30:30.666834 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:30:30.668765 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:30:30.669117 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:30:30.669198 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:30:30.672710 (+ 3512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:30:30.672714 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:30:30.672717 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:30:30.672749 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:30:30.674658 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:30:30.675427 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:30:30.675433 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:30:30.675436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:30:30.675535 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:30:30.675979 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:30:30.675984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:30:30.675986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:30:30.676269 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:30:30.676366 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 14:30:30.677102 (+ 736us) server_negotiation.cc:299] Negotiation successful
0808 14:30:30.677248 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 14:30:30.679706 19821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:30:30.660934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:33027 (local address 127.7.125.66:33783)
0808 14:30:30.662630 (+ 1696us) negotiation.cc:107] Waiting for socket to connect
0808 14:30:30.662696 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 14:30:30.663863 (+ 1167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:30:30.664698 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:30:30.664707 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:30:30.665079 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:30:30.665785 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:30:30.665805 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:30:30.667043 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:30:30.667046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:30:30.668618 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:30:30.668630 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:30:30.669437 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:30:30.669441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:30:30.669709 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:30:30.670555 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:30:30.670577 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:30:30.672470 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:30:30.674818 (+ 2348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:30:30.674824 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:30:30.674844 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:30:30.675279 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:30:30.675655 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:30:30.675659 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:30:30.675663 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:30:30.675863 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:30:30.676456 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:30:30.676466 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:30:30.676806 (+ 340us) client_negotiation.cc:769] Sending connection context
0808 14:30:30.677939 (+ 1133us) client_negotiation.cc:240] Negotiation successful
0808 14:30:30.678316 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1460,"thread_start_us":183,"threads_started":1}
I20260808 14:30:30.681424 19819 heartbeater.cc:347] Connected to a master server at 127.7.125.126:33027
I20260808 14:30:30.681682 19819 heartbeater.cc:464] Registering TS with master...
I20260808 14:30:30.682272 19819 heartbeater.cc:511] Master 127.7.125.126:33027 requested a full tablet report, sending...
I20260808 14:30:30.683544 19451 ts_manager.cc:194] Registered new tserver with Master: eb429499d109402cb2afc9f05caf93b5 (127.7.125.66:33647)
I20260808 14:30:30.684319 19451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:33783
WARNING: no policy specified for kudu/127.7.125.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
I20260808 14:30:30.717978 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.125.84:53237/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:33027
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:40137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:30:30.881047 19825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:30:30.881565 19825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:30:30.881688 19825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:30:30.887692 19825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:30:30.887912 19825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:30:30.888064 19825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:30:30.896845 19825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:40137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.125.84:53237/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:33027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:30:30.898623 19825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:30:30.900143 19825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:30:30.914126 19833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:30:30.915728 19825 server_base.cc:1034] running on GCE node
W20260808 14:30:30.916007 19830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:30:30.918047 19831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:30:30.919469 19825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:30:30.920601 19825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
Aug 08 14:30:30 dist-test-slave-w206 krb5kdc[18954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199430, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for HTTP/127.7.125.84@KRBTEST.COM
I20260808 14:30:30.928939 19825 hybrid_clock.cc:648] HybridClock initialized: now 1786199430928835 us; error 100 us; skew 500 ppm
Aug 08 14:30:30 dist-test-slave-w206 krb5kdc[18954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199430, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:30:30.933454 19825 init.cc:377] Logged in from keytab as kudu/127.7.125.67@KRBTEST.COM (short username kudu)
I20260808 14:30:30.934785 19825 webserver.cc:460] Webserver started at http://127.7.125.67:41437/ using document root <none> and password file <none>
I20260808 14:30:30.935773 19825 fs_manager.cc:359] Metadata directory not provided
I20260808 14:30:30.935947 19825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:30:30.936271 19825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 14:30:30 dist-test-slave-w206 krb5kdc[18954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199430, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for HTTP/127.7.125.84@KRBTEST.COM
I20260808 14:30:30.983431 19825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "6f1a9cfc0b8f4cbe81a904b9fc60b778"
format_stamp: "Formatted at 2026-08-08 14:30:30 on dist-test-slave-w206"
server_key: "6a3e378161c3b37208d98e0b0096a206"
server_key_iv: "6afb064ed31fb1b45823582bdc1193a0"
server_key_version: "kuduclusterkey@0"
I20260808 14:30:30.984423 19825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "6f1a9cfc0b8f4cbe81a904b9fc60b778"
format_stamp: "Formatted at 2026-08-08 14:30:30 on dist-test-slave-w206"
server_key: "6a3e378161c3b37208d98e0b0096a206"
server_key_iv: "6afb064ed31fb1b45823582bdc1193a0"
server_key_version: "kuduclusterkey@0"
I20260808 14:30:30.990583 19825 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260808 14:30:31.011786 19840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:30:31.013710 19825 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260808 14:30:31.013866 19825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "6f1a9cfc0b8f4cbe81a904b9fc60b778"
format_stamp: "Formatted at 2026-08-08 14:30:30 on dist-test-slave-w206"
server_key: "6a3e378161c3b37208d98e0b0096a206"
server_key_iv: "6afb064ed31fb1b45823582bdc1193a0"
server_key_version: "kuduclusterkey@0"
I20260808 14:30:31.014008 19825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:30:31.075803 19825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:30:31.085726 19825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:30:31.085960 19825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:30:31.086673 19825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:30:31.087774 19825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:30:31.087839 19825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:30:31.087901 19825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:30:31.087934 19825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:30:31.099558 19825 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:43333
I20260808 14:30:31.100847 19825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:30:31.103734 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 19825
I20260808 14:30:31.103893 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:30:31.104202 7669 external_mini_cluster.cc:1428] Setting key 40141dab4be9995822f3a4212abc882c
I20260808 14:30:31.109246 19955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:43333 every 8 connection(s)
Aug 08 14:30:31 dist-test-slave-w206 krb5kdc[18954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199430, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:30:31.125205 19679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:30:31.110357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:55551 (local address 127.7.125.126:33027)
0808 14:30:31.110499 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 14:30:31.110504 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:30:31.111637 (+ 1133us) server_negotiation.cc:407] Connection header received
0808 14:30:31.112900 (+ 1263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:30:31.112904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:30:31.112960 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:30:31.113014 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:30:31.114635 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.115508 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.117212 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.117558 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.117625 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:30:31.120780 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:30:31.120783 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:30:31.120786 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:30:31.120812 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:30:31.122707 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:30:31.123489 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:30:31.123493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:30:31.123495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:30:31.123587 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:30:31.123978 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:30:31.123981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:30:31.123983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:30:31.124189 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:30:31.124276 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 14:30:31.124932 (+ 656us) server_negotiation.cc:299] Negotiation successful
0808 14:30:31.125061 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 14:30:31.126995 19958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:30:31.110210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:33027 (local address 127.7.125.67:55551)
0808 14:30:31.111482 (+ 1272us) negotiation.cc:107] Waiting for socket to connect
0808 14:30:31.111539 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 14:30:31.112651 (+ 1112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:30:31.113448 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:30:31.113458 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:30:31.113825 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:30:31.114479 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:30:31.114496 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.115657 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.115660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:30:31.117075 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:30:31.117086 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.117821 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.117824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:30:31.118018 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:30:31.118727 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:30:31.118753 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:30:31.120595 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:30:31.122988 (+ 2393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:30:31.122994 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:30:31.123009 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:30:31.123374 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:30:31.123696 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:30:31.123699 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:30:31.123701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:30:31.123874 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:30:31.124343 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:30:31.124350 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:30:31.124658 (+ 308us) client_negotiation.cc:769] Sending connection context
0808 14:30:31.125585 (+ 927us) client_negotiation.cc:240] Negotiation successful
0808 14:30:31.125929 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1041,"thread_start_us":131,"threads_started":1}
I20260808 14:30:31.128463 19956 heartbeater.cc:347] Connected to a master server at 127.7.125.126:33027
I20260808 14:30:31.128742 19956 heartbeater.cc:464] Registering TS with master...
I20260808 14:30:31.129351 19956 heartbeater.cc:511] Master 127.7.125.126:33027 requested a full tablet report, sending...
I20260808 14:30:31.130436 19451 ts_manager.cc:194] Registered new tserver with Master: 6f1a9cfc0b8f4cbe81a904b9fc60b778 (127.7.125.67:43333)
I20260808 14:30:31.131204 19451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:55551
I20260808 14:30:31.132895 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:30:31.149297 19679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:30:31.139427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49152 (local address 127.7.125.126:33027)
0808 14:30:31.139587 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 14:30:31.139593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:30:31.140009 (+ 416us) server_negotiation.cc:407] Connection header received
0808 14:30:31.140227 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:30:31.140231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:30:31.140282 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:30:31.140335 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:30:31.141323 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.142162 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.143425 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.143755 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.143825 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:30:31.145682 (+ 1857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:30:31.145686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:30:31.145689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:30:31.145718 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:30:31.147429 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:30:31.147940 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:30:31.147944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:30:31.147946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:30:31.148030 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:30:31.148313 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:30:31.148316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:30:31.148318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:30:31.148530 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:30:31.148629 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 14:30:31.149013 (+ 384us) server_negotiation.cc:299] Negotiation successful
0808 14:30:31.149136 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 14:30:31.153097 19451 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49152:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:30:31.157481 19451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:30:31.176280 19971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:30:31.168652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:43333 (local address 127.0.0.1:48068)
0808 14:30:31.170223 (+ 1571us) negotiation.cc:107] Waiting for socket to connect
0808 14:30:31.170279 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 14:30:31.170414 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:30:31.171285 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:30:31.171292 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:30:31.171316 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:30:31.171462 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:30:31.171469 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.173053 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.173057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:30:31.174471 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:30:31.174482 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.175698 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.175703 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:30:31.175836 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:30:31.175889 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 14:30:31.175986 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 14:30:31.176110 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1382,"thread_start_us":114,"threads_started":1}
I20260808 14:30:31.178828 19973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:30:31.169808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:37429 (local address 127.0.0.1:35182)
0808 14:30:31.171942 (+ 2134us) negotiation.cc:107] Waiting for socket to connect
0808 14:30:31.171956 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:30:31.172041 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:30:31.173826 (+ 1785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:30:31.173829 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:30:31.173846 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:30:31.173931 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:30:31.173936 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.175265 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.175268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:30:31.177174 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:30:31.177185 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.178423 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.178428 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:30:31.178517 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:30:31.178539 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 14:30:31.178606 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 14:30:31.178672 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2069,"thread_start_us":88,"threads_started":1}
I20260808 14:30:31.179629 19972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:30:31.170201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35182 (local address 127.7.125.65:37429)
0808 14:30:31.172448 (+ 2247us) server_negotiation.cc:206] Beginning negotiation
0808 14:30:31.172454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:30:31.172477 (+ 23us) server_negotiation.cc:407] Connection header received
0808 14:30:31.172546 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:30:31.172551 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:30:31.173535 (+ 984us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:30:31.173672 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:30:31.174051 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.175138 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.177319 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.178110 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.178212 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:30:31.178358 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 14:30:31.179393 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0808 14:30:31.179503 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2154,"thread_start_us":194,"threads_started":1}
I20260808 14:30:31.181393 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:30:31.168830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48068 (local address 127.7.125.67:43333)
0808 14:30:31.170724 (+ 1894us) server_negotiation.cc:206] Beginning negotiation
0808 14:30:31.170742 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 14:30:31.170767 (+ 25us) server_negotiation.cc:407] Connection header received
0808 14:30:31.170856 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:30:31.170861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:30:31.171106 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:30:31.171616 (+ 510us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:30:31.171749 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.172912 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.174649 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.175546 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.180932 (+ 5386us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:30:31.181091 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 14:30:31.181147 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 14:30:31.181243 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1782,"thread_start_us":221,"threads_started":1}
I20260808 14:30:31.183168 19603 tablet_service.cc:1467] Processing CreateTablet for tablet ff192ce3c935490e9f73d656232fa66e (DEFAULT_TABLE table=test-table [id=835d05a36baa4d0dac1d3ee6dded4914]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:30:31.184441 19603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff192ce3c935490e9f73d656232fa66e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:30:31.184448 19889 tablet_service.cc:1467] Processing CreateTablet for tablet ff192ce3c935490e9f73d656232fa66e (DEFAULT_TABLE table=test-table [id=835d05a36baa4d0dac1d3ee6dded4914]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:30:31.185493 19889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff192ce3c935490e9f73d656232fa66e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:30:31.194591 19971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:30:31.171147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:33647 (local address 127.0.0.1:38724)
0808 14:30:31.176921 (+ 5774us) negotiation.cc:107] Waiting for socket to connect
0808 14:30:31.176936 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:30:31.177018 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:30:31.177808 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:30:31.177812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:30:31.177840 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:30:31.177914 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:30:31.177922 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.179346 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.179350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:30:31.181224 (+ 1874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:30:31.181236 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.194041 (+ 12805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.194047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:30:31.194161 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:30:31.194197 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 14:30:31.194259 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 14:30:31.194433 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5721,"thread_start_us":88,"threads_started":1}
I20260808 14:30:31.196177 19974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:30:31.171540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38724 (local address 127.7.125.66:33647)
0808 14:30:31.172263 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0808 14:30:31.172280 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:30:31.177130 (+ 4850us) server_negotiation.cc:407] Connection header received
0808 14:30:31.177254 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:30:31.177260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:30:31.177480 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:30:31.177648 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:30:31.178046 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.179208 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.181390 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.193844 (+ 12454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.195664 (+ 1820us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:30:31.195846 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 14:30:31.195917 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 14:30:31.196032 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":85,"threads_started":1}
I20260808 14:30:31.196714 19977 tablet_bootstrap.cc:492] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778: Bootstrap starting.
I20260808 14:30:31.199431 19738 tablet_service.cc:1467] Processing CreateTablet for tablet ff192ce3c935490e9f73d656232fa66e (DEFAULT_TABLE table=test-table [id=835d05a36baa4d0dac1d3ee6dded4914]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:30:31.200712 19738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff192ce3c935490e9f73d656232fa66e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:30:31.200944 19977 tablet_bootstrap.cc:654] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778: Neither blocks nor log segments found. Creating new log.
I20260808 14:30:31.201851 19977 log.cc:824] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778: Log is configured to *not* fsync() on all Append() calls
I20260808 14:30:31.202730 19976 tablet_bootstrap.cc:492] T ff192ce3c935490e9f73d656232fa66e P 060b9bcbfce147b39d37453cef4ff3f0: Bootstrap starting.
I20260808 14:30:31.205348 19976 tablet_bootstrap.cc:654] T ff192ce3c935490e9f73d656232fa66e P 060b9bcbfce147b39d37453cef4ff3f0: Neither blocks nor log segments found. Creating new log.
I20260808 14:30:31.207646 19976 log.cc:824] T ff192ce3c935490e9f73d656232fa66e P 060b9bcbfce147b39d37453cef4ff3f0: Log is configured to *not* fsync() on all Append() calls
I20260808 14:30:31.210196 19977 tablet_bootstrap.cc:492] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778: No bootstrap required, opened a new log
I20260808 14:30:31.210491 19977 ts_tablet_manager.cc:1397] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260808 14:30:31.213290 19976 tablet_bootstrap.cc:492] T ff192ce3c935490e9f73d656232fa66e P 060b9bcbfce147b39d37453cef4ff3f0: No bootstrap required, opened a new log
I20260808 14:30:31.213539 19976 ts_tablet_manager.cc:1397] T ff192ce3c935490e9f73d656232fa66e P 060b9bcbfce147b39d37453cef4ff3f0: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260808 14:30:31.215034 19977 raft_consensus.cc:348] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f1a9cfc0b8f4cbe81a904b9fc60b778" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 43333 } } peers { permanent_uuid: "060b9bcbfce147b39d37453cef4ff3f0" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37429 } } peers { permanent_uuid: "eb429499d109402cb2afc9f05caf93b5" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 33647 } }
I20260808 14:30:31.215509 19977 raft_consensus.cc:374] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:30:31.215629 19977 raft_consensus.cc:729] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f1a9cfc0b8f4cbe81a904b9fc60b778, State: Initialized, Role: FOLLOWER
I20260808 14:30:31.216214 19977 consensus_queue.cc:260] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f1a9cfc0b8f4cbe81a904b9fc60b778" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 43333 } } peers { permanent_uuid: "060b9bcbfce147b39d37453cef4ff3f0" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37429 } } peers { permanent_uuid: "eb429499d109402cb2afc9f05caf93b5" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 33647 } }
I20260808 14:30:31.218787 19977 ts_tablet_manager.cc:1428] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260808 14:30:31.219321 19956 heartbeater.cc:503] Master 127.7.125.126:33027 was elected leader, sending a full tablet report...
I20260808 14:30:31.219791 19980 tablet_bootstrap.cc:492] T ff192ce3c935490e9f73d656232fa66e P eb429499d109402cb2afc9f05caf93b5: Bootstrap starting.
I20260808 14:30:31.220147 19976 raft_consensus.cc:348] T ff192ce3c935490e9f73d656232fa66e P 060b9bcbfce147b39d37453cef4ff3f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f1a9cfc0b8f4cbe81a904b9fc60b778" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 43333 } } peers { permanent_uuid: "060b9bcbfce147b39d37453cef4ff3f0" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37429 } } peers { permanent_uuid: "eb429499d109402cb2afc9f05caf93b5" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 33647 } }
I20260808 14:30:31.220445 19976 raft_consensus.cc:374] T ff192ce3c935490e9f73d656232fa66e P 060b9bcbfce147b39d37453cef4ff3f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:30:31.220521 19976 raft_consensus.cc:729] T ff192ce3c935490e9f73d656232fa66e P 060b9bcbfce147b39d37453cef4ff3f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 060b9bcbfce147b39d37453cef4ff3f0, State: Initialized, Role: FOLLOWER
I20260808 14:30:31.221042 19976 consensus_queue.cc:260] T ff192ce3c935490e9f73d656232fa66e P 060b9bcbfce147b39d37453cef4ff3f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f1a9cfc0b8f4cbe81a904b9fc60b778" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 43333 } } peers { permanent_uuid: "060b9bcbfce147b39d37453cef4ff3f0" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37429 } } peers { permanent_uuid: "eb429499d109402cb2afc9f05caf93b5" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 33647 } }
I20260808 14:30:31.222187 19976 ts_tablet_manager.cc:1428] T ff192ce3c935490e9f73d656232fa66e P 060b9bcbfce147b39d37453cef4ff3f0: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260808 14:30:31.222667 19676 heartbeater.cc:503] Master 127.7.125.126:33027 was elected leader, sending a full tablet report...
I20260808 14:30:31.222944 19980 tablet_bootstrap.cc:654] T ff192ce3c935490e9f73d656232fa66e P eb429499d109402cb2afc9f05caf93b5: Neither blocks nor log segments found. Creating new log.
I20260808 14:30:31.223984 19980 log.cc:824] T ff192ce3c935490e9f73d656232fa66e P eb429499d109402cb2afc9f05caf93b5: Log is configured to *not* fsync() on all Append() calls
I20260808 14:30:31.226198 19980 tablet_bootstrap.cc:492] T ff192ce3c935490e9f73d656232fa66e P eb429499d109402cb2afc9f05caf93b5: No bootstrap required, opened a new log
I20260808 14:30:31.226394 19980 ts_tablet_manager.cc:1397] T ff192ce3c935490e9f73d656232fa66e P eb429499d109402cb2afc9f05caf93b5: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 14:30:31.231206 19980 raft_consensus.cc:348] T ff192ce3c935490e9f73d656232fa66e P eb429499d109402cb2afc9f05caf93b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f1a9cfc0b8f4cbe81a904b9fc60b778" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 43333 } } peers { permanent_uuid: "060b9bcbfce147b39d37453cef4ff3f0" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37429 } } peers { permanent_uuid: "eb429499d109402cb2afc9f05caf93b5" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 33647 } }
I20260808 14:30:31.231511 19980 raft_consensus.cc:374] T ff192ce3c935490e9f73d656232fa66e P eb429499d109402cb2afc9f05caf93b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:30:31.231559 19980 raft_consensus.cc:729] T ff192ce3c935490e9f73d656232fa66e P eb429499d109402cb2afc9f05caf93b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb429499d109402cb2afc9f05caf93b5, State: Initialized, Role: FOLLOWER
I20260808 14:30:31.232045 19980 consensus_queue.cc:260] T ff192ce3c935490e9f73d656232fa66e P eb429499d109402cb2afc9f05caf93b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f1a9cfc0b8f4cbe81a904b9fc60b778" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 43333 } } peers { permanent_uuid: "060b9bcbfce147b39d37453cef4ff3f0" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37429 } } peers { permanent_uuid: "eb429499d109402cb2afc9f05caf93b5" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 33647 } }
I20260808 14:30:31.233171 19980 ts_tablet_manager.cc:1428] T ff192ce3c935490e9f73d656232fa66e P eb429499d109402cb2afc9f05caf93b5: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 14:30:31.233569 19819 heartbeater.cc:503] Master 127.7.125.126:33027 was elected leader, sending a full tablet report...
W20260808 14:30:31.261008 19677 tablet.cc:2369] T ff192ce3c935490e9f73d656232fa66e P 060b9bcbfce147b39d37453cef4ff3f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 14:30:31 dist-test-slave-w206 krb5kdc[18954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199430, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for HTTP/127.7.125.84@KRBTEST.COM
W20260808 14:30:31.357667 19957 tablet.cc:2369] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:30:31.409307 19820 tablet.cc:2369] T ff192ce3c935490e9f73d656232fa66e P eb429499d109402cb2afc9f05caf93b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:30:31.481240 19981 raft_consensus.cc:482] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:30:31.481505 19981 raft_consensus.cc:504] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f1a9cfc0b8f4cbe81a904b9fc60b778" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 43333 } } peers { permanent_uuid: "060b9bcbfce147b39d37453cef4ff3f0" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37429 } } peers { permanent_uuid: "eb429499d109402cb2afc9f05caf93b5" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 33647 } }
I20260808 14:30:31.482755 19981 leader_election.cc:288] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 060b9bcbfce147b39d37453cef4ff3f0 (127.7.125.65:37429), eb429499d109402cb2afc9f05caf93b5 (127.7.125.66:33647)
I20260808 14:30:31.488622 19958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:30:31.483032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:37429 (local address 127.7.125.67:54929)
0808 14:30:31.483535 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0808 14:30:31.483550 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:30:31.483657 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:30:31.484259 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:30:31.484261 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:30:31.484280 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:30:31.484404 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:30:31.484413 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.485486 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.485489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:30:31.487025 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:30:31.487038 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.488017 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.488021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:30:31.488256 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:30:31.488319 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 14:30:31.488394 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 14:30:31.488482 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":194}
I20260808 14:30:31.488629 19974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:30:31.483841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:49377 (local address 127.7.125.66:33647)
0808 14:30:31.483965 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 14:30:31.483970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:30:31.483983 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:30:31.484029 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:30:31.484032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:30:31.484073 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:30:31.484136 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:30:31.484580 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.485343 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.487181 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.487799 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.487892 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:30:31.487950 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 14:30:31.488458 (+ 508us) server_negotiation.cc:299] Negotiation successful
0808 14:30:31.488530 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 14:30:31.489027 19987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:30:31.483307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:33647 (local address 127.7.125.67:49377)
0808 14:30:31.483720 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0808 14:30:31.483734 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:30:31.483809 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:30:31.484258 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:30:31.484261 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:30:31.484332 (+ 71us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:30:31.484446 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:30:31.484455 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.485496 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.485499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:30:31.487039 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:30:31.487045 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.488046 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.488049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:30:31.488260 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:30:31.488319 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 14:30:31.488763 (+ 444us) client_negotiation.cc:240] Negotiation successful
0808 14:30:31.488855 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":262,"spinlock_wait_cycles":38272,"thread_start_us":154,"threads_started":1}
I20260808 14:30:31.489800 19753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff192ce3c935490e9f73d656232fa66e" candidate_uuid: "6f1a9cfc0b8f4cbe81a904b9fc60b778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb429499d109402cb2afc9f05caf93b5" is_pre_election: true
I20260808 14:30:31.489929 19972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:30:31.483105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:54929 (local address 127.7.125.65:37429)
0808 14:30:31.483254 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 14:30:31.483259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:30:31.483945 (+ 686us) server_negotiation.cc:407] Connection header received
0808 14:30:31.484014 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:30:31.484017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:30:31.484065 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:30:31.484136 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:30:31.484550 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.485346 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.487175 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.487777 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.487912 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:30:31.487954 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 14:30:31.489705 (+ 1751us) server_negotiation.cc:299] Negotiation successful
0808 14:30:31.489803 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 14:30:31.490159 19753 raft_consensus.cc:2454] T ff192ce3c935490e9f73d656232fa66e P eb429499d109402cb2afc9f05caf93b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f1a9cfc0b8f4cbe81a904b9fc60b778 in term 0.
I20260808 14:30:31.490447 19626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff192ce3c935490e9f73d656232fa66e" candidate_uuid: "6f1a9cfc0b8f4cbe81a904b9fc60b778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "060b9bcbfce147b39d37453cef4ff3f0" is_pre_election: true
I20260808 14:30:31.490743 19626 raft_consensus.cc:2454] T ff192ce3c935490e9f73d656232fa66e P 060b9bcbfce147b39d37453cef4ff3f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f1a9cfc0b8f4cbe81a904b9fc60b778 in term 0.
I20260808 14:30:31.491266 19842 leader_election.cc:302] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 060b9bcbfce147b39d37453cef4ff3f0, 6f1a9cfc0b8f4cbe81a904b9fc60b778; no voters:
I20260808 14:30:31.491601 19981 raft_consensus.cc:2781] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:30:31.491693 19981 raft_consensus.cc:482] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:30:31.491736 19981 raft_consensus.cc:3037] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:30:31.493057 19981 raft_consensus.cc:504] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f1a9cfc0b8f4cbe81a904b9fc60b778" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 43333 } } peers { permanent_uuid: "060b9bcbfce147b39d37453cef4ff3f0" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37429 } } peers { permanent_uuid: "eb429499d109402cb2afc9f05caf93b5" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 33647 } }
I20260808 14:30:31.493403 19981 leader_election.cc:288] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778 [CANDIDATE]: Term 1 election: Requested vote from peers 060b9bcbfce147b39d37453cef4ff3f0 (127.7.125.65:37429), eb429499d109402cb2afc9f05caf93b5 (127.7.125.66:33647)
I20260808 14:30:31.493882 19753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff192ce3c935490e9f73d656232fa66e" candidate_uuid: "6f1a9cfc0b8f4cbe81a904b9fc60b778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb429499d109402cb2afc9f05caf93b5"
I20260808 14:30:31.493986 19626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff192ce3c935490e9f73d656232fa66e" candidate_uuid: "6f1a9cfc0b8f4cbe81a904b9fc60b778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "060b9bcbfce147b39d37453cef4ff3f0"
I20260808 14:30:31.494040 19753 raft_consensus.cc:3037] T ff192ce3c935490e9f73d656232fa66e P eb429499d109402cb2afc9f05caf93b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:30:31.494107 19626 raft_consensus.cc:3037] T ff192ce3c935490e9f73d656232fa66e P 060b9bcbfce147b39d37453cef4ff3f0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:30:31.495438 19753 raft_consensus.cc:2454] T ff192ce3c935490e9f73d656232fa66e P eb429499d109402cb2afc9f05caf93b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f1a9cfc0b8f4cbe81a904b9fc60b778 in term 1.
I20260808 14:30:31.495533 19626 raft_consensus.cc:2454] T ff192ce3c935490e9f73d656232fa66e P 060b9bcbfce147b39d37453cef4ff3f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f1a9cfc0b8f4cbe81a904b9fc60b778 in term 1.
I20260808 14:30:31.496021 19842 leader_election.cc:302] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f1a9cfc0b8f4cbe81a904b9fc60b778, eb429499d109402cb2afc9f05caf93b5; no voters:
I20260808 14:30:31.496377 19981 raft_consensus.cc:2781] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:30:31.496654 19981 raft_consensus.cc:686] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778 [term 1 LEADER]: Becoming Leader. State: Replica: 6f1a9cfc0b8f4cbe81a904b9fc60b778, State: Running, Role: LEADER
I20260808 14:30:31.497027 19981 consensus_queue.cc:237] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f1a9cfc0b8f4cbe81a904b9fc60b778" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 43333 } } peers { permanent_uuid: "060b9bcbfce147b39d37453cef4ff3f0" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37429 } } peers { permanent_uuid: "eb429499d109402cb2afc9f05caf93b5" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 33647 } }
I20260808 14:30:31.500582 19452 catalog_manager.cc:5495] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f1a9cfc0b8f4cbe81a904b9fc60b778 (127.7.125.67). New cstate: current_term: 1 leader_uuid: "6f1a9cfc0b8f4cbe81a904b9fc60b778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f1a9cfc0b8f4cbe81a904b9fc60b778" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 43333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "060b9bcbfce147b39d37453cef4ff3f0" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb429499d109402cb2afc9f05caf93b5" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 33647 } health_report { overall_health: UNKNOWN } } }
I20260808 14:30:31.540304 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:30:31.535937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48070 (local address 127.7.125.67:43333)
0808 14:30:31.536101 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 14:30:31.536106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:30:31.536125 (+ 19us) server_negotiation.cc:407] Connection header received
0808 14:30:31.536221 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:30:31.536224 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:30:31.536300 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:30:31.536428 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:30:31.536668 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.537539 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.538431 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:30:31.538818 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:30:31.538937 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:30:31.539297 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:30:31.539870 (+ 573us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:30:31.540033 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 14:30:31.540070 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 14:30:31.540131 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 14:30:31.546655 19626 raft_consensus.cc:1261] T ff192ce3c935490e9f73d656232fa66e P 060b9bcbfce147b39d37453cef4ff3f0 [term 1 FOLLOWER]: Refusing update from remote peer 6f1a9cfc0b8f4cbe81a904b9fc60b778: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:30:31.546667 19753 raft_consensus.cc:1261] T ff192ce3c935490e9f73d656232fa66e P eb429499d109402cb2afc9f05caf93b5 [term 1 FOLLOWER]: Refusing update from remote peer 6f1a9cfc0b8f4cbe81a904b9fc60b778: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:30:31.547622 19988 consensus_queue.cc:1035] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "060b9bcbfce147b39d37453cef4ff3f0" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:30:31.547883 19981 consensus_queue.cc:1035] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb429499d109402cb2afc9f05caf93b5" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 33647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:30:31.558676 19999 mvcc.cc:205] Tried to move back new op lower bound from 7316272871607291904 to 7316272871415029760. Current Snapshot: MvccSnapshot[applied={T|T < 7316272871607291904}]
I20260808 14:30:31.559330 20000 mvcc.cc:205] Tried to move back new op lower bound from 7316272871607291904 to 7316272871415029760. Current Snapshot: MvccSnapshot[applied={T|T < 7316272871607291904}]
I20260808 14:30:31.567648 19452 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:30:31.567839 19452 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:30:31.570116 19452 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5734b27b820240babf70d56cfdf18428: Sending DeleteTablet for 3 replicas of tablet ff192ce3c935490e9f73d656232fa66e
I20260808 14:30:31.570829 19889 tablet_service.cc:1514] Processing DeleteTablet for tablet ff192ce3c935490e9f73d656232fa66e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:30:31 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:48068
I20260808 14:30:31.570838 19603 tablet_service.cc:1514] Processing DeleteTablet for tablet ff192ce3c935490e9f73d656232fa66e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:30:31 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:35182
I20260808 14:30:31.571246 19738 tablet_service.cc:1514] Processing DeleteTablet for tablet ff192ce3c935490e9f73d656232fa66e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:30:31 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:38724
I20260808 14:30:31.571895 20010 tablet_replica.cc:330] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778: stopping tablet replica
I20260808 14:30:31.572204 20010 raft_consensus.cc:2229] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:30:31.572530 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 19537
I20260808 14:30:31.573428 20010 raft_consensus.cc:2258] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:30:31.575677 20010 ts_tablet_manager.cc:1905] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:30:31.579461 20010 ts_tablet_manager.cc:1918] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:30:31.579577 20010 log.cc:1196] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/ff192ce3c935490e9f73d656232fa66e
I20260808 14:30:31.579953 20010 ts_tablet_manager.cc:1939] T ff192ce3c935490e9f73d656232fa66e P 6f1a9cfc0b8f4cbe81a904b9fc60b778: Deleting consensus metadata
I20260808 14:30:31.581375 19436 catalog_manager.cc:4841] TS 6f1a9cfc0b8f4cbe81a904b9fc60b778 (127.7.125.67:43333): tablet ff192ce3c935490e9f73d656232fa66e (table test-table [id=835d05a36baa4d0dac1d3ee6dded4914]) successfully deleted
W20260808 14:30:31.582809 19437 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.125.65:37429 (error 108)
I20260808 14:30:31.583058 20011 tablet_replica.cc:330] T ff192ce3c935490e9f73d656232fa66e P eb429499d109402cb2afc9f05caf93b5: stopping tablet replica
I20260808 14:30:31.583356 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 19684
I20260808 14:30:31.591845 19971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:30:31.591407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:37429 (local address 127.0.0.1:35184)
0808 14:30:31.591627 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 14:30:31.591714 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.65:37429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260808 14:30:31.592052 19437 catalog_manager.cc:4584] TS 060b9bcbfce147b39d37453cef4ff3f0 (127.7.125.65:37429): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ff192ce3c935490e9f73d656232fa66e: Network error: Client connection negotiation failed: client connection to 127.7.125.65:37429: connect: Connection refused (error 111)
I20260808 14:30:31.597064 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 19825
I20260808 14:30:31.598431 19971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:30:31.598055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:33647 (local address 127.0.0.1:38732)
0808 14:30:31.598267 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 14:30:31.598327 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.66:33647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260808 14:30:31.605628 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 19418
I20260808 14:30:31.613864 7669 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 14:30:31.959983 7669 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 14:30:31.960165 7669 mini_ranger.cc:66] Stopping Ranger...
I20260808 14:30:32.306165 7669 mini_ranger.cc:68] Stopped Ranger
2026-08-08 14:30:32.306 UTC [18984] LOG: received smart shutdown request
2026-08-08 14:30:32.310 UTC [18984] LOG: background worker "logical replication launcher" (PID 18993) exited with exit code 1
2026-08-08 14:30:32.311 UTC [18988] LOG: shutting down
2026-08-08 14:30:32.496 UTC [18984] LOG: database system is shut down
2026-08-08T14:30:32Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49120 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:30:32 dist-test-slave-w206 krb5kdc[20021](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:30:32 dist-test-slave-w206 krb5kdc[20021](info): set up 2 sockets
Aug 08 14:30:32 dist-test-slave-w206 krb5kdc[20021](info): commencing operation
krb5kdc: starting...
W20260808 14:30:33.699703 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:30:33 dist-test-slave-w206 krb5kdc[20021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:30:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:30:33Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.125.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.125.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.125.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/rangeradmin_127.7.125.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.125.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/rangeradmin_127.7.125.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.125.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.125.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.125.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/rangerlookup_127.7.125.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.125.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/rangerlookup_127.7.125.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.125.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.125.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.125.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/HTTP_127.7.125.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.125.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/HTTP_127.7.125.84@KRBTEST.COM.keytab.
I20260808 14:30:33.894280 7669 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskngm_Z6/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/postgres -l logfile start
2026-08-08 14:30:36.497 UTC [20052] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 14:30:36.497 UTC [20052] LOG: listening on IPv4 address "127.7.125.84", port 56537
2026-08-08 14:30:36.500 UTC [20052] LOG: listening on Unix socket "/tmp/.s.PGSQL.56537"
2026-08-08 14:30:36.522 UTC [20055] LOG: database system was shut down at 2026-08-08 14:30:34 UTC
2026-08-08 14:30:36.525 UTC [20052] LOG: database system is ready to accept connections
I20260808 14:30:37.511823 7669 mini_postgres.cc:96] Postgres bound to 56537
2026-08-08 14:30:37.519 UTC [20066] FATAL: database "slave" does not exist
127.7.125.84:56537 - accepting connections
I20260808 14:30:37.519740 7669 mini_ranger.cc:161] Starting Ranger...
I20260808 14:30:37.536525 7669 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 14:30:38.041173 7669 mini_ranger.cc:87] Created ranger Postgres database
I20260808 14:30:38.041271 7669 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ranger-admin
2026-08-08 14:30:38,332 [I] DB FLAVOR :POSTGRES
2026-08-08 14:30:38,332 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 14:30:38,332 [I] Checking connection..
2026-08-08 14:30:38,332 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:56537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 14:30:38,568 [I] Checking connection passed.
2026-08-08 14:30:38,568 [I] --------- Verifying version history table ---------
2026-08-08 14:30:38,568 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:56537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 14:30:38,842 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 14:30:38,842 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:56537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 14:30:39,074 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 14:30:39,074 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 14:30:39,074 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:56537/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 14:30:39.335 UTC [20133] WARNING: there is no transaction in progress
2026-08-08 14:30:39,342 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 14:30:39,343 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:56537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 14:30:39,582 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 14:30:39,582 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 14:30:39,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:56537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 14:30:39,813 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:56537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 14:30:40,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:56537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-w206.c.gcp-upstream.internal','N') ;"
2026-08-08 14:30:40,312 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:56537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 14:30:40,555 [I] Table x_portal_user does not exist in database ranger
2026-08-08 14:30:40,556 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:56537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 14:30:40,828 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 14:30:40,828 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:56537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 14:30:41,087 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:56537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 14:30:41,342 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 14:30:41,342 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:56537/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 14:30:42.402 UTC [20250] WARNING: there is no transaction in progress
2026-08-08 14:30:42.420 UTC [20250] WARNING: there is no transaction in progress
2026-08-08 14:30:42.439 UTC [20250] WARNING: there is no transaction in progress
2026-08-08 14:30:42.457 UTC [20250] WARNING: there is no transaction in progress
2026-08-08 14:30:42.475 UTC [20250] WARNING: there is no transaction in progress
2026-08-08 14:30:42.497 UTC [20250] WARNING: there is no transaction in progress
2026-08-08 14:30:42.564 UTC [20250] WARNING: there is no transaction in progress
2026-08-08 14:30:42.572 UTC [20250] WARNING: there is no transaction in progress
2026-08-08 14:30:42.588 UTC [20250] WARNING: there is no transaction in progress
2026-08-08 14:30:42.599 UTC [20250] WARNING: there is no transaction in progress
2026-08-08 14:30:42.610 UTC [20250] WARNING: there is no transaction in progress
2026-08-08 14:30:42.625 UTC [20250] WARNING: there is no transaction in progress
2026-08-08 14:30:42.635 UTC [20250] WARNING: there is no transaction in progress
2026-08-08 14:30:42.643 UTC [20250] WARNING: there is no transaction in progress
2026-08-08 14:30:42.651 UTC [20250] WARNING: there is no transaction in progress
2026-08-08 14:30:42.659 UTC [20250] WARNING: there is no transaction in progress
2026-08-08 14:30:43,423 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 14:30:43,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:56537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 14:30:43,695 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 14:30:43,695 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:56537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 14:30:43,951 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 14:30:43,951 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:56537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 14:30:44,218 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 14:30:44,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:56537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 14:30:44,485 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:56537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 14:30:44,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:56537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-w206.c.gcp-upstream.internal';"
2026-08-08 14:30:44,986 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 14:30:44,987 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:56537/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 14:30:45,245 [I] DB_PATCHES have already been applied
I20260808 14:30:45.251056 7669 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ranger-admin:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskngm_Z6/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 14:30:45.251127 7669 mini_ranger.cc:193] Using host: 127.7.125.84
I20260808 14:30:45.255642 7669 mini_ranger.cc:227] Ranger admin URL: http://127.7.125.84:34833
Aug 08, 2026 2:30:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:30:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:30:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 2:30:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 2:30:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 2:30:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 2:30:46 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 14:30:46.414592 7669 mini_ranger.cc:160] Time spent starting Ranger: real 8.895s user 0.004s sys 0.009s
Aug 08, 2026 2:31:06 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 2:31:06 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 08, 2026 2:31:06 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 2:31:06 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 2:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 2:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 2:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 2:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 2:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 2:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 2:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 2:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 2:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 2:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 2:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 2:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 2:31:06 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 2:31:07 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 14:31:08.474188 7669 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.125.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.125.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.125.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/rangerkms_127.7.125.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.125.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/rangerkms_127.7.125.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.125.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.125.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.125.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/HTTP_127.7.125.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.125.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/HTTP_127.7.125.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 14:31:08 dist-test-slave-w206 krb5kdc[20021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199468, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 14:31:08.574702 7669 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 14:31:08.586014 7669 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 14:31:09.512177 7669 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 14:31:09.512280 7669 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ranger-kms
I20260808 14:31:09.512303 7669 mini_ranger_kms.cc:227] Using postgres at 127.7.125.84:56537
2026-08-08 14:31:09,706 [I] DB FLAVOR :POSTGRES
2026-08-08 14:31:09,706 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 14:31:09,706 [I] Checking connection
2026-08-08 14:31:09,707 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:56537/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 14:31:09,976 [I] connection success
2026-08-08 14:31:09,976 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 14:31:09,976 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 14:31:09,976 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:56537/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 14:31:10,218 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 14:31:10,218 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 14:31:10,218 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 14:31:10,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.125.84:56537/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 14:31:10,483 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 14:31:10.973593 7669 mini_ranger_kms.cc:310] Created kms service
I20260808 14:31:11.194041 7669 mini_ranger_kms.cc:326] Created kudu user
I20260808 14:31:11.240890 7669 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 14:31:11 dist-test-slave-w206 krb5kdc[20021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199468, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.125.84@KRBTEST.COM
I20260808 14:31:11.644441 7669 mini_ranger_kms.cc:384] Added ranger policy
I20260808 14:31:11.644609 7669 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ranger-kms:/tmp/dist-test-taskngm_Z6/build/debug/bin/postgresql.jar:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskngm_Z6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 14:31:11.644634 7669 mini_ranger_kms.cc:240] Using host: 127.7.125.84
I20260808 14:31:11.649315 7669 mini_ranger_kms.cc:276] Ranger KMS PID: 20443
I20260808 14:31:11.649420 7669 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.125.84:38191
Aug 08, 2026 2:31:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:31:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:31:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 2:31:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 2:31:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 2:31:13 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 14:31:14.039582 7669 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.465s user 0.024s sys 0.154s
I20260808 14:31:14.039747 7669 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 14:31:14.039839 7669 mini_ranger_kms.cc:401] 127.7.125.84:38191/kms/v1/keys
Aug 08 14:31:14 dist-test-slave-w206 krb5kdc[20021](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786199474, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.125.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 14:31:14 dist-test-slave-w206 krb5kdc[20021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199474, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.125.84@KRBTEST.COM for HTTP/127.7.125.84@KRBTEST.COM
Aug 08 14:31:14 dist-test-slave-w206 krb5kdc[20021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199474, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.125.84@KRBTEST.COM for HTTP/127.7.125.84@KRBTEST.COM
Aug 08 14:31:17 dist-test-slave-w206 krb5kdc[20021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:31:17.388144 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:44433
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:41493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:44433
--ranger_config_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.125.84:38191/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:31:17.507480 20497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:17.507804 20497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:17.507851 20497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:17.512082 20497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:31:17.512171 20497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:17.512190 20497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:17.512207 20497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:31:17.512223 20497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:31:17.518208 20497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:41493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.125.84:38191/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:44433
--ranger_config_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:44433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:17.519519 20497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:17.520645 20497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:17.531141 20503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:17.531237 20506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:17.531141 20504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:17.531978 20497 server_base.cc:1034] running on GCE node
I20260808 14:31:17.532645 20497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:17.534198 20497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:17.535398 20497 hybrid_clock.cc:648] HybridClock initialized: now 1786199477535378 us; error 39 us; skew 500 ppm
Aug 08 14:31:17 dist-test-slave-w206 krb5kdc[20021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199477, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:17.538528 20497 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:31:17.539621 20497 webserver.cc:460] Webserver started at http://127.7.125.126:44547/ using document root <none> and password file <none>
I20260808 14:31:17.540227 20497 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:17.540287 20497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:17.540536 20497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 14:31:17 dist-test-slave-w206 krb5kdc[20021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199477, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for HTTP/127.7.125.84@KRBTEST.COM
I20260808 14:31:17.591141 20497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "426aefb2448b43c780ef838b429197dd"
format_stamp: "Formatted at 2026-08-08 14:31:17 on dist-test-slave-w206"
server_key: "062bec5a4ee86d4f22f27b1d309ba79d"
server_key_iv: "71695bf26c685ecd355d927a2b2e1d65"
server_key_version: "kuduclusterkey@0"
I20260808 14:31:17.591922 20497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "426aefb2448b43c780ef838b429197dd"
format_stamp: "Formatted at 2026-08-08 14:31:17 on dist-test-slave-w206"
server_key: "062bec5a4ee86d4f22f27b1d309ba79d"
server_key_iv: "71695bf26c685ecd355d927a2b2e1d65"
server_key_version: "kuduclusterkey@0"
I20260808 14:31:17.597195 20497 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 14:31:17.613272 20513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:17.614759 20497 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:31:17.614925 20497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "426aefb2448b43c780ef838b429197dd"
format_stamp: "Formatted at 2026-08-08 14:31:17 on dist-test-slave-w206"
server_key: "062bec5a4ee86d4f22f27b1d309ba79d"
server_key_iv: "71695bf26c685ecd355d927a2b2e1d65"
server_key_version: "kuduclusterkey@0"
I20260808 14:31:17.615134 20497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:17.638357 20497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:17.641880 20497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:17.642102 20497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:17.651269 20497 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:44433
I20260808 14:31:17.651312 20568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:44433 every 8 connection(s)
I20260808 14:31:17.652344 20497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:31:17.655942 20569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:31:17.657718 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 20497
I20260808 14:31:17.657886 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:31:17.658183 7669 external_mini_cluster.cc:1428] Setting key 2c01c67064c2476508d851371ab18db7
I20260808 14:31:17.663249 20569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd: Bootstrap starting.
I20260808 14:31:17.666117 20569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd: Neither blocks nor log segments found. Creating new log.
I20260808 14:31:17.667063 20569 log.cc:824] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:31:17 dist-test-slave-w206 krb5kdc[20021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:31:17.669524 20569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd: No bootstrap required, opened a new log
I20260808 14:31:17.673426 20569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426aefb2448b43c780ef838b429197dd" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44433 } }
I20260808 14:31:17.673705 20569 raft_consensus.cc:374] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:31:17.673712 20572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:17.659878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48576 (local address 127.7.125.126:44433)
0808 14:31:17.660360 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:17.660378 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:17.660415 (+ 37us) server_negotiation.cc:407] Connection header received
0808 14:31:17.661063 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:17.661080 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:17.661420 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:17.661765 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:17.662653 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:17.663796 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:17.664726 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:17.665211 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:17.665402 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:17.668309 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:17.668315 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:17.668330 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:17.668358 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:17.670677 (+ 2319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:17.671235 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:17.671246 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:17.671251 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:17.671359 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:17.671660 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:17.671663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:17.671665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:17.672071 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:17.672284 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:17.672588 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 14:31:17.672914 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":160,"threads_started":1}
I20260808 14:31:17.673835 20569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 426aefb2448b43c780ef838b429197dd, State: Initialized, Role: FOLLOWER
I20260808 14:31:17.674424 20569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426aefb2448b43c780ef838b429197dd" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44433 } }
I20260808 14:31:17.674580 20569 raft_consensus.cc:388] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:31:17.674665 20569 raft_consensus.cc:482] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:31:17.674770 20569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:31:17.675938 20569 raft_consensus.cc:504] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426aefb2448b43c780ef838b429197dd" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44433 } }
I20260808 14:31:17.676384 20569 leader_election.cc:302] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 426aefb2448b43c780ef838b429197dd; no voters:
I20260808 14:31:17.676738 20569 leader_election.cc:288] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:31:17.676832 20574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:31:17.677098 20574 raft_consensus.cc:686] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [term 1 LEADER]: Becoming Leader. State: Replica: 426aefb2448b43c780ef838b429197dd, State: Running, Role: LEADER
I20260808 14:31:17.677474 20574 consensus_queue.cc:237] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426aefb2448b43c780ef838b429197dd" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44433 } }
I20260808 14:31:17.677991 20569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:31:17.678711 20574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 426aefb2448b43c780ef838b429197dd. Latest consensus state: current_term: 1 leader_uuid: "426aefb2448b43c780ef838b429197dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426aefb2448b43c780ef838b429197dd" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44433 } } }
I20260808 14:31:17.678941 20574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [sys.catalog]: This master's current role is: LEADER
I20260808 14:31:17.679255 20576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "426aefb2448b43c780ef838b429197dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426aefb2448b43c780ef838b429197dd" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44433 } } }
I20260808 14:31:17.679388 20576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [sys.catalog]: This master's current role is: LEADER
I20260808 14:31:17.680258 20569 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 14:31:17.684542 20583 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:31:17.689847 20583 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:31:17.701300 20583 catalog_manager.cc:1347] Generated new cluster ID: 506ae5c61fd54167af4f53eb22ad5d97
I20260808 14:31:17.701498 20583 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:31:17.711553 20583 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:31:17.713145 20583 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:31:17.721266 20583 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd: Generated new TSK 0
I20260808 14:31:17.722369 20583 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.125.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
I20260808 14:31:18.558506 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.125.84:38191/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:44433
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:41493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:31:18.745707 20614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:18.746172 20614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:18.746261 20614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:18.753423 20614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:18.753584 20614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:18.753726 20614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:31:18.762320 20614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:41493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.125.84:38191/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:44433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:18.764067 20614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:18.765532 20614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:18.779672 20623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:18.779672 20622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:18.780323 20625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:18.780643 20614 server_base.cc:1034] running on GCE node
I20260808 14:31:18.781188 20614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:18.782033 20614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:18.783252 20614 hybrid_clock.cc:648] HybridClock initialized: now 1786199478783228 us; error 49 us; skew 500 ppm
Aug 08 14:31:18 dist-test-slave-w206 krb5kdc[20021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199478, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:18.787392 20614 init.cc:377] Logged in from keytab as kudu/127.7.125.65@KRBTEST.COM (short username kudu)
I20260808 14:31:18.788674 20614 webserver.cc:460] Webserver started at http://127.7.125.65:33941/ using document root <none> and password file <none>
I20260808 14:31:18.789841 20614 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:18.790141 20614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:18.790562 20614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 14:31:18 dist-test-slave-w206 krb5kdc[20021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199478, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for HTTP/127.7.125.84@KRBTEST.COM
I20260808 14:31:18.826519 20614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "cc88cbf66de2433182856da91d317164"
format_stamp: "Formatted at 2026-08-08 14:31:18 on dist-test-slave-w206"
server_key: "0ece29430826de912ffaa11a5c20108e"
server_key_iv: "05227d3dbeb3138d48fd6ab723d0d1b5"
server_key_version: "kuduclusterkey@0"
I20260808 14:31:18.827402 20614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "cc88cbf66de2433182856da91d317164"
format_stamp: "Formatted at 2026-08-08 14:31:18 on dist-test-slave-w206"
server_key: "0ece29430826de912ffaa11a5c20108e"
server_key_iv: "05227d3dbeb3138d48fd6ab723d0d1b5"
server_key_version: "kuduclusterkey@0"
I20260808 14:31:18.835731 20614 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.001s
I20260808 14:31:18.860774 20635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:18.862398 20614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 14:31:18.862545 20614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "cc88cbf66de2433182856da91d317164"
format_stamp: "Formatted at 2026-08-08 14:31:18 on dist-test-slave-w206"
server_key: "0ece29430826de912ffaa11a5c20108e"
server_key_iv: "05227d3dbeb3138d48fd6ab723d0d1b5"
server_key_version: "kuduclusterkey@0"
I20260808 14:31:18.862694 20614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:18.892977 20614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:18.896597 20614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:18.896797 20614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:18.897394 20614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:31:18.898454 20614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:31:18.898515 20614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:18.898577 20614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:31:18.898603 20614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:18.910164 20614 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:35399
I20260808 14:31:18.914474 20754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:35399 every 8 connection(s)
I20260808 14:31:18.915035 20614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:31:18.921998 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 20614
I20260808 14:31:18.922130 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:31:18.922431 7669 external_mini_cluster.cc:1428] Setting key 24e40369220cf4bb05d08b30760a3aa4
Aug 08 14:31:18 dist-test-slave-w206 krb5kdc[20021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199478, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:31:18.938511 20757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:18.917531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:42901 (local address 127.7.125.126:44433)
0808 14:31:18.920945 (+ 3414us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:18.920950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:18.920965 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:31:18.921010 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:18.921013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:18.921062 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:18.921148 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:18.927926 (+ 6778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:18.928764 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:18.930502 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:18.930851 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:18.930946 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:18.934082 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:18.934086 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:18.934089 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:18.934119 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:18.936001 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:18.936626 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:18.936629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:18.936632 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:18.936733 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:18.937116 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:18.937119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:18.937121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:18.937336 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:18.937420 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:18.938142 (+ 722us) server_negotiation.cc:299] Negotiation successful
0808 14:31:18.938351 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3325,"thread_start_us":115,"threads_started":1}
I20260808 14:31:18.942185 20758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:18.918027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:44433 (local address 127.7.125.65:42901)
0808 14:31:18.919337 (+ 1310us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:18.919403 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:18.920472 (+ 1069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:18.921460 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:18.921470 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:18.926666 (+ 5196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:18.927518 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:18.927541 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:18.928927 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:18.928931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:18.930334 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:18.930346 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:18.931169 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:18.931173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:18.931383 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:18.932111 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:18.932133 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:18.933871 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:18.936146 (+ 2275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:18.936152 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:18.936170 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:18.936512 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:18.936830 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:18.936833 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:18.936835 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:18.937013 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:18.937472 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:18.937478 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:18.937813 (+ 335us) client_negotiation.cc:769] Sending connection context
0808 14:31:18.938016 (+ 203us) client_negotiation.cc:240] Negotiation successful
0808 14:31:18.938354 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1083,"thread_start_us":150,"threads_started":1}
I20260808 14:31:18.944088 20755 heartbeater.cc:347] Connected to a master server at 127.7.125.126:44433
I20260808 14:31:18.944341 20755 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:18.944923 20755 heartbeater.cc:511] Master 127.7.125.126:44433 requested a full tablet report, sending...
I20260808 14:31:18.946718 20530 ts_manager.cc:194] Registered new tserver with Master: cc88cbf66de2433182856da91d317164 (127.7.125.65:35399)
I20260808 14:31:18.948552 20530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:42901
WARNING: no policy specified for kudu/127.7.125.66@KRBTEST.COM; defaulting to no policy
Aug 08 14:31:18 dist-test-slave-w206 krb5kdc[20021](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786199478, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
I20260808 14:31:18.981835 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.125.84:38191/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:44433
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:41493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:31:19.186198 20761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:19.186657 20761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:19.186757 20761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:19.193892 20761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:19.194000 20761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:19.194119 20761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:31:19.203838 20761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:41493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.125.84:38191/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:44433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:19.205440 20761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:19.206924 20761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:31:19.221913 20761 server_base.cc:1034] running on GCE node
W20260808 14:31:19.222116 20770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:19.222345 20768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:19.222589 20767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:19.223163 20761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:19.224009 20761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:19.225226 20761 hybrid_clock.cc:648] HybridClock initialized: now 1786199479225198 us; error 53 us; skew 500 ppm
Aug 08 14:31:19 dist-test-slave-w206 krb5kdc[20021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199479, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:19.229067 20761 init.cc:377] Logged in from keytab as kudu/127.7.125.66@KRBTEST.COM (short username kudu)
I20260808 14:31:19.230161 20761 webserver.cc:460] Webserver started at http://127.7.125.66:36507/ using document root <none> and password file <none>
I20260808 14:31:19.230863 20761 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:19.230950 20761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:19.231204 20761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 14:31:19 dist-test-slave-w206 krb5kdc[20021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199479, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for HTTP/127.7.125.84@KRBTEST.COM
I20260808 14:31:19.247340 20761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "0d0be99a4c7848a3828ee93f614f7cdb"
format_stamp: "Formatted at 2026-08-08 14:31:19 on dist-test-slave-w206"
server_key: "3d2d584eee633380487809673be5498b"
server_key_iv: "2c46cb13ef07bdc0dfaa98509534c407"
server_key_version: "kuduclusterkey@0"
I20260808 14:31:19.248231 20761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "0d0be99a4c7848a3828ee93f614f7cdb"
format_stamp: "Formatted at 2026-08-08 14:31:19 on dist-test-slave-w206"
server_key: "3d2d584eee633380487809673be5498b"
server_key_iv: "2c46cb13ef07bdc0dfaa98509534c407"
server_key_version: "kuduclusterkey@0"
I20260808 14:31:19.255084 20761 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260808 14:31:19.272940 20778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:19.274912 20761 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260808 14:31:19.275085 20761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "0d0be99a4c7848a3828ee93f614f7cdb"
format_stamp: "Formatted at 2026-08-08 14:31:19 on dist-test-slave-w206"
server_key: "3d2d584eee633380487809673be5498b"
server_key_iv: "2c46cb13ef07bdc0dfaa98509534c407"
server_key_version: "kuduclusterkey@0"
I20260808 14:31:19.275231 20761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:19.296818 20761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:19.300124 20761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:19.300282 20761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:19.300926 20761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:31:19.301795 20761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:31:19.301877 20761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:19.301947 20761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:31:19.302001 20761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:19.313463 20761 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:40391
I20260808 14:31:19.314719 20761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:31:19.315313 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 20761
I20260808 14:31:19.315459 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:31:19.315766 7669 external_mini_cluster.cc:1428] Setting key 17077264c44919aa6252234d11cf63a1
I20260808 14:31:19.322394 20896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:40391 every 8 connection(s)
WARNING: no policy specified for kudu/127.7.125.67@KRBTEST.COM; defaulting to no policy
Aug 08 14:31:19 dist-test-slave-w206 krb5kdc[20021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199479, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.67@KRBTEST.COM" created.
I20260808 14:31:19.346704 20757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:19.327776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:41277 (local address 127.7.125.126:44433)
0808 14:31:19.327923 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:19.327928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:19.328288 (+ 360us) server_negotiation.cc:407] Connection header received
0808 14:31:19.329764 (+ 1476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:19.329769 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:19.329840 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:19.329913 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:19.332027 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:19.332900 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:19.337179 (+ 4279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:19.337590 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:19.337683 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:19.341116 (+ 3433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:19.341120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:19.341124 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:19.341156 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:19.343449 (+ 2293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:19.344336 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:19.344341 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:19.344343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:19.344457 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:19.345028 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:19.345033 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:19.345035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:19.345330 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:19.345436 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:19.346291 (+ 855us) server_negotiation.cc:299] Negotiation successful
0808 14:31:19.346462 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 14:31:19.348692 20899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:19.326546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:44433 (local address 127.7.125.66:41277)
0808 14:31:19.328054 (+ 1508us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:19.328130 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:19.329452 (+ 1322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:19.330437 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:19.330447 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:19.330885 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:19.331806 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:19.331832 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:19.333109 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:19.333116 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:19.334790 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:19.334805 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:19.337939 (+ 3134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:19.337943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:19.338191 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:19.338967 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:19.339026 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:19.340869 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:19.343681 (+ 2812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:19.343690 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:19.343713 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:19.344160 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:19.344606 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:19.344609 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:19.344612 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:19.344848 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:19.345522 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:19.345530 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:19.345914 (+ 384us) client_negotiation.cc:769] Sending connection context
0808 14:31:19.347039 (+ 1125us) client_negotiation.cc:240] Negotiation successful
0808 14:31:19.347475 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1239,"thread_start_us":188,"threads_started":1}
I20260808 14:31:19.350265 20897 heartbeater.cc:347] Connected to a master server at 127.7.125.126:44433
I20260808 14:31:19.350560 20897 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:19.351231 20897 heartbeater.cc:511] Master 127.7.125.126:44433 requested a full tablet report, sending...
I20260808 14:31:19.352581 20530 ts_manager.cc:194] Registered new tserver with Master: 0d0be99a4c7848a3828ee93f614f7cdb (127.7.125.66:40391)
I20260808 14:31:19.353353 20530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:41277
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
I20260808 14:31:19.375447 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.125.84:38191/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:44433
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:41493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 14:31:19 dist-test-slave-w206 krb5kdc[20021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199478, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for HTTP/127.7.125.84@KRBTEST.COM
W20260808 14:31:19.546697 20902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:19.547133 20902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:19.547245 20902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:19.553669 20902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:19.553817 20902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:19.553939 20902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:31:19.561822 20902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:41493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.125.84:38191/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:44433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:19.563498 20902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:19.564891 20902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:19.578480 20908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:19.579088 20907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:19.578545 20910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:19.580430 20902 server_base.cc:1034] running on GCE node
I20260808 14:31:19.581065 20902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:19.581864 20902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:19.583104 20902 hybrid_clock.cc:648] HybridClock initialized: now 1786199479583039 us; error 83 us; skew 500 ppm
Aug 08 14:31:19 dist-test-slave-w206 krb5kdc[20021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199479, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:19.586865 20902 init.cc:377] Logged in from keytab as kudu/127.7.125.67@KRBTEST.COM (short username kudu)
I20260808 14:31:19.588079 20902 webserver.cc:460] Webserver started at http://127.7.125.67:40405/ using document root <none> and password file <none>
I20260808 14:31:19.588829 20902 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:19.588967 20902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:19.589282 20902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 14:31:19 dist-test-slave-w206 krb5kdc[20021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199479, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for HTTP/127.7.125.84@KRBTEST.COM
I20260808 14:31:19.606389 20902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "c6ebcbc987fe4fc5ac7c214938bf1a40"
format_stamp: "Formatted at 2026-08-08 14:31:19 on dist-test-slave-w206"
server_key: "85b09f0a08a00c5226685adce8fe2458"
server_key_iv: "33abd1f17426f5b09a8b2be526ecd1c4"
server_key_version: "kuduclusterkey@0"
I20260808 14:31:19.615715 20902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "c6ebcbc987fe4fc5ac7c214938bf1a40"
format_stamp: "Formatted at 2026-08-08 14:31:19 on dist-test-slave-w206"
server_key: "85b09f0a08a00c5226685adce8fe2458"
server_key_iv: "33abd1f17426f5b09a8b2be526ecd1c4"
server_key_version: "kuduclusterkey@0"
I20260808 14:31:19.621937 20902 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 14:31:19.638305 20919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:19.639967 20902 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260808 14:31:19.640125 20902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "c6ebcbc987fe4fc5ac7c214938bf1a40"
format_stamp: "Formatted at 2026-08-08 14:31:19 on dist-test-slave-w206"
server_key: "85b09f0a08a00c5226685adce8fe2458"
server_key_iv: "33abd1f17426f5b09a8b2be526ecd1c4"
server_key_version: "kuduclusterkey@0"
I20260808 14:31:19.640290 20902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:19.665587 20902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:19.669695 20902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:19.669917 20902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:19.670648 20902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:31:19.671751 20902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:31:19.671818 20902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:19.671875 20902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:31:19.671952 20902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260808 14:31:19.683806 20902 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:37837
I20260808 14:31:19.683925 21039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:37837 every 8 connection(s)
I20260808 14:31:19.685169 20902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:31:19.686092 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 20902
I20260808 14:31:19.686235 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:31:19.686584 7669 external_mini_cluster.cc:1428] Setting key af9ab520228a26780c4270f6c2d40e72
Aug 08 14:31:19 dist-test-slave-w206 krb5kdc[20021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199479, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:31:19.705534 20757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:19.688948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:39435 (local address 127.7.125.126:44433)
0808 14:31:19.689087 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:19.689091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:19.689467 (+ 376us) server_negotiation.cc:407] Connection header received
0808 14:31:19.690760 (+ 1293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:19.690764 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:19.690820 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:19.690947 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:19.692853 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:19.693673 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:19.695461 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:19.695807 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:19.695909 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:19.701059 (+ 5150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:19.701063 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:19.701067 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:19.701096 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:19.703008 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:19.703677 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:19.703681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:19.703683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:19.703776 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:19.704210 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:19.704214 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:19.704216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:19.704458 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:19.704562 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:19.705229 (+ 667us) server_negotiation.cc:299] Negotiation successful
0808 14:31:19.705367 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 14:31:19.706609 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:19.687921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:44433 (local address 127.7.125.67:39435)
0808 14:31:19.689286 (+ 1365us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:19.689358 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:19.690517 (+ 1159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:19.691286 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:19.691296 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:19.691671 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:19.692658 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:19.692681 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:19.693859 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:19.693862 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:19.695310 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:19.695321 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:19.696134 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:19.696139 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:19.696352 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:19.697131 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:19.697153 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:19.698958 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:19.703118 (+ 4160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:19.703125 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:19.703144 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:19.703536 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:19.703884 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:19.703887 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:19.703888 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:19.704079 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:19.704582 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:19.704589 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:19.704922 (+ 333us) client_negotiation.cc:769] Sending connection context
0808 14:31:19.705126 (+ 204us) client_negotiation.cc:240] Negotiation successful
0808 14:31:19.705478 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1143,"thread_start_us":157,"threads_started":1}
I20260808 14:31:19.707841 21040 heartbeater.cc:347] Connected to a master server at 127.7.125.126:44433
I20260808 14:31:19.708114 21040 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:19.708774 21040 heartbeater.cc:511] Master 127.7.125.126:44433 requested a full tablet report, sending...
I20260808 14:31:19.710011 20530 ts_manager.cc:194] Registered new tserver with Master: c6ebcbc987fe4fc5ac7c214938bf1a40 (127.7.125.67:37837)
I20260808 14:31:19.710783 20530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:39435
I20260808 14:31:19.712486 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:31:19.713248 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 20614
I20260808 14:31:19.727140 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 20761
I20260808 14:31:19.735275 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 20902
I20260808 14:31:19.742621 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 20497
I20260808 14:31:19.751413 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:44433
--webserver_interface=127.7.125.126
--webserver_port=44547
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:41493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:44433
--ranger_config_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.125.84:38191/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.125.84:38191/kms with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:31:19.868098 21043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:19.868391 21043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:19.868430 21043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:19.872512 21043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:31:19.872581 21043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:19.872602 21043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:19.872618 21043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:31:19.872634 21043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:31:19.878407 21043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:41493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.125.84:38191/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:44433
--ranger_config_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:44433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=44547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:19.879555 21043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:19.880649 21043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:31:19.889513 21043 server_base.cc:1034] running on GCE node
W20260808 14:31:19.889477 21048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:19.889505 21049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:19.889477 21051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:19.890358 21043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:19.891605 21043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:19.892822 21043 hybrid_clock.cc:648] HybridClock initialized: now 1786199479892800 us; error 35 us; skew 500 ppm
Aug 08 14:31:19 dist-test-slave-w206 krb5kdc[20021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199479, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:19.895763 21043 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:31:19.896749 21043 webserver.cc:460] Webserver started at http://127.7.125.126:44547/ using document root <none> and password file <none>
I20260808 14:31:19.897316 21043 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:19.897390 21043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 14:31:19 dist-test-slave-w206 krb5kdc[20021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199479, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for HTTP/127.7.125.84@KRBTEST.COM
I20260808 14:31:19.980201 21043 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 14:31:19.982667 21060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:19.984129 21043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 14:31:19.984272 21043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "426aefb2448b43c780ef838b429197dd"
format_stamp: "Formatted at 2026-08-08 14:31:17 on dist-test-slave-w206"
server_key: "062bec5a4ee86d4f22f27b1d309ba79d"
server_key_iv: "71695bf26c685ecd355d927a2b2e1d65"
server_key_version: "kuduclusterkey@0"
I20260808 14:31:19.984860 21043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:20.015311 21043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:20.018275 21043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:20.018491 21043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:20.026427 21043 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:44433
I20260808 14:31:20.026458 21116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:44433 every 8 connection(s)
I20260808 14:31:20.027583 21043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:31:20.031095 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 21043
I20260808 14:31:20.031587 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.125.84:38191/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:35399
--local_ip_for_outbound_sockets=127.7.125.65
--tserver_master_addrs=127.7.125.126:44433
--webserver_port=33941
--webserver_interface=127.7.125.65
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:41493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.125.84:38191/kms with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:31:20.032186 21117 sys_catalog.cc:263] Verifying existing consensus state
I20260808 14:31:20.033888 21117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd: Bootstrap starting.
I20260808 14:31:20.051213 21117 log.cc:824] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd: Log is configured to *not* fsync() on all Append() calls
I20260808 14:31:20.058324 21117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:31:20.058768 21117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd: Bootstrap complete.
I20260808 14:31:20.063199 21117 raft_consensus.cc:348] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426aefb2448b43c780ef838b429197dd" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44433 } }
I20260808 14:31:20.063647 21117 raft_consensus.cc:729] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 426aefb2448b43c780ef838b429197dd, State: Initialized, Role: FOLLOWER
I20260808 14:31:20.064157 21117 consensus_queue.cc:260] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426aefb2448b43c780ef838b429197dd" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44433 } }
I20260808 14:31:20.064314 21117 raft_consensus.cc:388] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:31:20.064405 21117 raft_consensus.cc:482] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:31:20.064522 21117 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [term 1 FOLLOWER]: Advancing to term 2
I20260808 14:31:20.066538 21117 raft_consensus.cc:504] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426aefb2448b43c780ef838b429197dd" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44433 } }
I20260808 14:31:20.066932 21117 leader_election.cc:302] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 426aefb2448b43c780ef838b429197dd; no voters:
I20260808 14:31:20.067366 21117 leader_election.cc:288] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 14:31:20.067437 21121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [term 2 FOLLOWER]: Leader election won for term 2
I20260808 14:31:20.067660 21121 raft_consensus.cc:686] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [term 2 LEADER]: Becoming Leader. State: Replica: 426aefb2448b43c780ef838b429197dd, State: Running, Role: LEADER
I20260808 14:31:20.068043 21121 consensus_queue.cc:237] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426aefb2448b43c780ef838b429197dd" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44433 } }
I20260808 14:31:20.068701 21117 sys_catalog.cc:564] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:31:20.069523 21122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "426aefb2448b43c780ef838b429197dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426aefb2448b43c780ef838b429197dd" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44433 } } }
I20260808 14:31:20.069669 21122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [sys.catalog]: This master's current role is: LEADER
I20260808 14:31:20.070056 21123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 426aefb2448b43c780ef838b429197dd. Latest consensus state: current_term: 2 leader_uuid: "426aefb2448b43c780ef838b429197dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426aefb2448b43c780ef838b429197dd" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 44433 } } }
I20260808 14:31:20.070232 21123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd [sys.catalog]: This master's current role is: LEADER
I20260808 14:31:20.070930 21129 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:31:20.071218 21117 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 14:31:20.077364 21129 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:31:20.078383 21129 catalog_manager.cc:1259] Loaded cluster ID: 506ae5c61fd54167af4f53eb22ad5d97
I20260808 14:31:20.078477 21129 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:31:20.081669 21129 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:31:20.082412 21129 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd: Loaded TSK: 0
I20260808 14:31:20.083701 21129 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 14:31:20.188593 21119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:20.189011 21119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:20.189081 21119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:20.195914 21119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:20.196012 21119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:20.196105 21119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:31:20.204607 21119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:41493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.125.84:38191/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:35399
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=33941
--tserver_master_addrs=127.7.125.126:44433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:20.206164 21119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:20.207660 21119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:20.221566 21156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:20.221908 21119 server_base.cc:1034] running on GCE node
W20260808 14:31:20.221844 21158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:20.222276 21155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:20.222839 21119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:20.223680 21119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:20.224898 21119 hybrid_clock.cc:648] HybridClock initialized: now 1786199480224874 us; error 38 us; skew 500 ppm
Aug 08 14:31:20 dist-test-slave-w206 krb5kdc[20021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:20.228924 21119 init.cc:377] Logged in from keytab as kudu/127.7.125.65@KRBTEST.COM (short username kudu)
I20260808 14:31:20.230055 21119 webserver.cc:460] Webserver started at http://127.7.125.65:33941/ using document root <none> and password file <none>
I20260808 14:31:20.230727 21119 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:20.230824 21119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 14:31:20 dist-test-slave-w206 krb5kdc[20021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for HTTP/127.7.125.84@KRBTEST.COM
I20260808 14:31:20.264835 21119 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.003s sys 0.000s
I20260808 14:31:20.268671 21166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:20.271968 21119 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260808 14:31:20.272130 21119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "cc88cbf66de2433182856da91d317164"
format_stamp: "Formatted at 2026-08-08 14:31:18 on dist-test-slave-w206"
server_key: "0ece29430826de912ffaa11a5c20108e"
server_key_iv: "05227d3dbeb3138d48fd6ab723d0d1b5"
server_key_version: "kuduclusterkey@0"
I20260808 14:31:20.273015 21119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:20.288770 21119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:20.292775 21119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:20.293007 21119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:20.293772 21119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:31:20.294886 21119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:31:20.294957 21119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:20.295053 21119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:31:20.295090 21119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:20.311232 21284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:35399 every 8 connection(s)
I20260808 14:31:20.311421 21119 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:35399
I20260808 14:31:20.312786 21119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:31:20.322271 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 21119
I20260808 14:31:20.322839 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.125.84:38191/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:40391
--local_ip_for_outbound_sockets=127.7.125.66
--tserver_master_addrs=127.7.125.126:44433
--webserver_port=36507
--webserver_interface=127.7.125.66
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:41493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.125.84:38191/kms with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 14:31:20 dist-test-slave-w206 krb5kdc[20021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:31:20.342617 21288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:20.315495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:34337 (local address 127.7.125.126:44433)
0808 14:31:20.316180 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:20.316204 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:20.316771 (+ 567us) server_negotiation.cc:407] Connection header received
0808 14:31:20.318328 (+ 1557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:20.318344 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:20.318762 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:20.319144 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:20.320980 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:20.323702 (+ 2722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:20.325140 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:20.325965 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:20.326201 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:20.333741 (+ 7540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:20.333749 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:20.333772 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:20.333823 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:20.336446 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:20.337309 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:20.337317 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:20.337324 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:20.337496 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:20.337982 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:20.337985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:20.337987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:20.338563 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:20.338710 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:20.341150 (+ 2440us) server_negotiation.cc:299] Negotiation successful
0808 14:31:20.341473 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":141,"threads_started":1}
I20260808 14:31:20.343266 21289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:20.315561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:44433 (local address 127.7.125.65:34337)
0808 14:31:20.316541 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:20.316600 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:20.317706 (+ 1106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:20.319670 (+ 1964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:20.319679 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:20.320087 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:20.320762 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:20.320786 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:20.323893 (+ 3107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:20.323897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:20.324955 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:20.324973 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:20.326198 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:20.326204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:20.326448 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:20.331257 (+ 4809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:20.331285 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:20.333373 (+ 2088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:20.336642 (+ 3269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:20.336651 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:20.336669 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:20.337126 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:20.337625 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:20.337629 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:20.337631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:20.337852 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:20.338787 (+ 935us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:20.338795 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:20.339287 (+ 492us) client_negotiation.cc:769] Sending connection context
0808 14:31:20.339517 (+ 230us) client_negotiation.cc:240] Negotiation successful
0808 14:31:20.339911 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"thread_start_us":171,"threads_started":1}
I20260808 14:31:20.346172 21286 heartbeater.cc:347] Connected to a master server at 127.7.125.126:44433
I20260808 14:31:20.346459 21286 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:20.347134 21286 heartbeater.cc:511] Master 127.7.125.126:44433 requested a full tablet report, sending...
W20260808 14:31:20.349881 21286 heartbeater.cc:644] Failed to heartbeat to 127.7.125.126:44433 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 14:31:20.350051 21286 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:20.350286 21286 heartbeater.cc:511] Master 127.7.125.126:44433 requested a full tablet report, sending...
I20260808 14:31:20.351255 21286 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:20.351445 21286 heartbeater.cc:511] Master 127.7.125.126:44433 requested a full tablet report, sending...
W20260808 14:31:20.352265 21286 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 14:31:20.489296 21290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:20.489655 21290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:20.489740 21290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:20.494375 21290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:20.494462 21290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:20.494565 21290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:31:20.501228 21290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:41493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.125.84:38191/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:40391
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=36507
--tserver_master_addrs=127.7.125.126:44433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:20.502547 21290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:20.503722 21290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:20.514781 21295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:20.514762 21296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:20.516006 21299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:20.516418 21290 server_base.cc:1034] running on GCE node
I20260808 14:31:20.516899 21290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:20.517668 21290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:20.518914 21290 hybrid_clock.cc:648] HybridClock initialized: now 1786199480518882 us; error 42 us; skew 500 ppm
Aug 08 14:31:20 dist-test-slave-w206 krb5kdc[20021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:20.522643 21290 init.cc:377] Logged in from keytab as kudu/127.7.125.66@KRBTEST.COM (short username kudu)
I20260808 14:31:20.523936 21290 webserver.cc:460] Webserver started at http://127.7.125.66:36507/ using document root <none> and password file <none>
I20260808 14:31:20.524724 21290 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:20.524884 21290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 14:31:20 dist-test-slave-w206 krb5kdc[20021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for HTTP/127.7.125.84@KRBTEST.COM
I20260808 14:31:20.556224 21290 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.002s sys 0.000s
I20260808 14:31:20.559888 21308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:20.561766 21290 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260808 14:31:20.561959 21290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "0d0be99a4c7848a3828ee93f614f7cdb"
format_stamp: "Formatted at 2026-08-08 14:31:19 on dist-test-slave-w206"
server_key: "3d2d584eee633380487809673be5498b"
server_key_iv: "2c46cb13ef07bdc0dfaa98509534c407"
server_key_version: "kuduclusterkey@0"
I20260808 14:31:20.562806 21290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:20.589637 21290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:20.596590 21290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:20.596841 21290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:20.597640 21290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:31:20.598879 21290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:31:20.598948 21290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:20.599133 21290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:31:20.599170 21290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:20.612246 21290 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:40391
I20260808 14:31:20.613690 21290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:31:20.614003 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 21290
I20260808 14:31:20.614513 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.125.84:38191/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:37837
--local_ip_for_outbound_sockets=127.7.125.67
--tserver_master_addrs=127.7.125.126:44433
--webserver_port=40405
--webserver_interface=127.7.125.67
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:41493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.125.84:38191/kms with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:31:20.620450 21431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:40391 every 8 connection(s)
Aug 08 14:31:20 dist-test-slave-w206 krb5kdc[20021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:31:20.647972 21288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:20.622504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:37895 (local address 127.7.125.126:44433)
0808 14:31:20.622668 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:20.622673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:20.627674 (+ 5001us) server_negotiation.cc:407] Connection header received
0808 14:31:20.629471 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:20.629476 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:20.629543 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:20.629603 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:20.631674 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:20.632649 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:20.635168 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:20.635599 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:20.635679 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:20.641436 (+ 5757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:20.641441 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:20.641445 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:20.641480 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:20.643830 (+ 2350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:20.645842 (+ 2012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:20.645849 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:20.645851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:20.645961 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:20.646434 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:20.646441 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:20.646442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:20.646729 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:20.646835 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:20.647621 (+ 786us) server_negotiation.cc:299] Negotiation successful
0808 14:31:20.647791 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 14:31:20.657510 21436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:20.623179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:44433 (local address 127.7.125.66:37895)
0808 14:31:20.627406 (+ 4227us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:20.627491 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:20.629128 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:20.630124 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:20.630135 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:20.630666 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:20.631464 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:20.631488 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:20.632837 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:20.632840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:20.635005 (+ 2165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:20.635018 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:20.635933 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:20.635937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:20.636399 (+ 462us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:20.637208 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:20.637233 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:20.641151 (+ 3918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:20.645159 (+ 4008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:20.645167 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:20.645189 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:20.645664 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:20.646090 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:20.646093 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:20.646096 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:20.646306 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:20.646909 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:20.646916 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:20.647279 (+ 363us) client_negotiation.cc:769] Sending connection context
0808 14:31:20.648542 (+ 1263us) client_negotiation.cc:240] Negotiation successful
0808 14:31:20.649240 (+ 698us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3913,"thread_start_us":153,"threads_started":1}
I20260808 14:31:20.660038 21432 heartbeater.cc:347] Connected to a master server at 127.7.125.126:44433
I20260808 14:31:20.660321 21432 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:20.660966 21432 heartbeater.cc:511] Master 127.7.125.126:44433 requested a full tablet report, sending...
W20260808 14:31:20.662895 21432 heartbeater.cc:644] Failed to heartbeat to 127.7.125.126:44433 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 14:31:20.663154 21432 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:20.663368 21432 heartbeater.cc:511] Master 127.7.125.126:44433 requested a full tablet report, sending...
I20260808 14:31:20.664394 21432 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:20.664578 21432 heartbeater.cc:511] Master 127.7.125.126:44433 requested a full tablet report, sending...
W20260808 14:31:20.665679 21432 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 14:31:20.779955 21435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:20.780277 21435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:20.780377 21435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:20.785959 21435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:20.786056 21435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:20.786165 21435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:31:20.793861 21435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:41493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.125.84:38191/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:37837
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=40405
--tserver_master_addrs=127.7.125.126:44433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:20.795267 21435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:20.796602 21435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:20.807368 21442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:20.807356 21441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:20.807483 21444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:20.807582 21435 server_base.cc:1034] running on GCE node
I20260808 14:31:20.808281 21435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:20.809129 21435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:20.810341 21435 hybrid_clock.cc:648] HybridClock initialized: now 1786199480810328 us; error 58 us; skew 500 ppm
Aug 08 14:31:20 dist-test-slave-w206 krb5kdc[20021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:20.815394 21435 init.cc:377] Logged in from keytab as kudu/127.7.125.67@KRBTEST.COM (short username kudu)
I20260808 14:31:20.816488 21435 webserver.cc:460] Webserver started at http://127.7.125.67:40405/ using document root <none> and password file <none>
I20260808 14:31:20.817262 21435 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:20.817428 21435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 14:31:20 dist-test-slave-w206 krb5kdc[20021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for HTTP/127.7.125.84@KRBTEST.COM
I20260808 14:31:20.845705 21435 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:31:20.849103 21452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:20.850812 21435 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260808 14:31:20.850968 21435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "c6ebcbc987fe4fc5ac7c214938bf1a40"
format_stamp: "Formatted at 2026-08-08 14:31:19 on dist-test-slave-w206"
server_key: "85b09f0a08a00c5226685adce8fe2458"
server_key_iv: "33abd1f17426f5b09a8b2be526ecd1c4"
server_key_version: "kuduclusterkey@0"
I20260808 14:31:20.851737 21435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:20.868542 21435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:20.872081 21435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:20.872325 21435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:20.873040 21435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:31:20.874102 21435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:31:20.874178 21435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:20.874246 21435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:31:20.874310 21435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:20.887703 21435 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:37837
I20260808 14:31:20.887734 21575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:37837 every 8 connection(s)
I20260808 14:31:20.888873 21435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:31:20.892450 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 21435
Aug 08 14:31:20 dist-test-slave-w206 krb5kdc[20021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:31:20.904670 21288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:20.891028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:51411 (local address 127.7.125.126:44433)
0808 14:31:20.891171 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:20.891176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:20.892216 (+ 1040us) server_negotiation.cc:407] Connection header received
0808 14:31:20.893499 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:20.893502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:20.893549 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:20.893652 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:20.895166 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:20.895838 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:20.897146 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:20.897392 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:20.897499 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:20.900383 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:20.900388 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:20.900395 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:20.900417 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:20.902338 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:20.903038 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:20.903042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:20.903043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:20.903138 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:20.903547 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:20.903550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:20.903551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:20.903723 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:20.903834 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:20.904373 (+ 539us) server_negotiation.cc:299] Negotiation successful
0808 14:31:20.904496 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44,"spinlock_wait_cycles":1920}
I20260808 14:31:20.905766 21579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:20.891503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:44433 (local address 127.7.125.67:51411)
0808 14:31:20.892043 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:20.892104 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:20.893248 (+ 1144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:20.893841 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:20.893848 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:20.894217 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:20.894880 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:20.894943 (+ 63us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:20.895983 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:20.895987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:20.897017 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:20.897026 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:20.897554 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:20.897558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:20.897739 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:20.898440 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:20.898457 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:20.900207 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:20.902479 (+ 2272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:20.902488 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:20.902509 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:20.902883 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:20.903215 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:20.903219 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:20.903221 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:20.903423 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:20.903834 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:20.903842 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:20.904120 (+ 278us) client_negotiation.cc:769] Sending connection context
0808 14:31:20.904319 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 14:31:20.904648 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":113,"threads_started":1}
I20260808 14:31:20.907049 21576 heartbeater.cc:347] Connected to a master server at 127.7.125.126:44433
I20260808 14:31:20.907341 21576 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:20.907886 21576 heartbeater.cc:511] Master 127.7.125.126:44433 requested a full tablet report, sending...
W20260808 14:31:20.909554 21576 heartbeater.cc:644] Failed to heartbeat to 127.7.125.126:44433 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 14:31:20.909690 21576 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:20.909869 21576 heartbeater.cc:511] Master 127.7.125.126:44433 requested a full tablet report, sending...
I20260808 14:31:20.910713 21576 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:20.910946 21576 heartbeater.cc:511] Master 127.7.125.126:44433 requested a full tablet report, sending...
W20260808 14:31:20.911696 21576 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 14:31:20.912602 21581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:20.893110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48592 (local address 127.7.125.126:44433)
0808 14:31:20.893710 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:20.893714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:20.894022 (+ 308us) server_negotiation.cc:407] Connection header received
0808 14:31:20.894149 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:20.894152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:20.894201 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:20.894295 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:20.895582 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:20.896285 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:20.898599 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:20.898940 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:20.899094 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:20.900923 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:20.900928 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:20.900931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:20.900982 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:20.903079 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:20.908189 (+ 5110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:20.908193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:20.908195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:20.908320 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:20.908905 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:20.908909 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:20.908911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:20.909137 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:20.909355 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:20.912295 (+ 2940us) server_negotiation.cc:299] Negotiation successful
0808 14:31:20.912430 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"spinlock_wait_cycles":1792,"thread_start_us":101,"threads_started":1}
Aug 08 14:31:21 dist-test-slave-w206 krb5kdc[20021](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786199481, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:21.352538 21286 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:21.352885 21286 heartbeater.cc:511] Master 127.7.125.126:44433 requested a full tablet report, sending...
I20260808 14:31:21.354236 21078 ts_manager.cc:194] Registered new tserver with Master: cc88cbf66de2433182856da91d317164 (127.7.125.65:35399)
I20260808 14:31:21.355830 21078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:34337
Aug 08 14:31:21 dist-test-slave-w206 krb5kdc[20021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199481, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for HTTP/127.7.125.84@KRBTEST.COM
I20260808 14:31:21.665899 21432 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:21.666220 21432 heartbeater.cc:511] Master 127.7.125.126:44433 requested a full tablet report, sending...
I20260808 14:31:21.666934 21078 ts_manager.cc:194] Registered new tserver with Master: 0d0be99a4c7848a3828ee93f614f7cdb (127.7.125.66:40391)
I20260808 14:31:21.667685 21078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:37895
I20260808 14:31:21.911928 21576 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:21.912230 21576 heartbeater.cc:511] Master 127.7.125.126:44433 requested a full tablet report, sending...
I20260808 14:31:21.913208 21078 ts_manager.cc:194] Registered new tserver with Master: c6ebcbc987fe4fc5ac7c214938bf1a40 (127.7.125.67:37837)
Aug 08 14:31:21 dist-test-slave-w206 krb5kdc[20021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199481, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for HTTP/127.7.125.84@KRBTEST.COM
I20260808 14:31:21.913810 21078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:51411
I20260808 14:31:21.927224 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:31:21.948483 21601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:21.929497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48606 (local address 127.7.125.126:44433)
0808 14:31:21.929760 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:21.929767 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:21.929780 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:31:21.929818 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:21.929821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:21.929877 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:21.929980 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:21.930883 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:21.931729 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:21.932876 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:21.933206 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:21.933403 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:21.936817 (+ 3414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:21.936822 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:21.936825 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:21.936857 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:21.942294 (+ 5437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:21.943403 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:21.943407 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:21.943409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:21.943517 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:21.944265 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:21.944269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:21.944271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:21.944868 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:21.945399 (+ 531us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:21.948145 (+ 2746us) server_negotiation.cc:299] Negotiation successful
0808 14:31:21.948305 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":82,"threads_started":1}
I20260808 14:31:21.953174 21078 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48606:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:31:21.958211 21078 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:31:21.982728 21607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:21.975938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:35399 (local address 127.0.0.1:59898)
0808 14:31:21.976323 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:21.976376 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:21.976499 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:21.977705 (+ 1206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:21.977712 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:21.977738 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:31:21.977926 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:21.977937 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:21.979564 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:21.979568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:21.980923 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:21.980932 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:21.982196 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:21.982200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:21.982314 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:21.982354 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 14:31:21.982456 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 14:31:21.982568 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":122,"threads_started":1}
I20260808 14:31:21.985405 21608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:21.976888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59898 (local address 127.7.125.65:35399)
0808 14:31:21.977175 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:21.977182 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:21.977208 (+ 26us) server_negotiation.cc:407] Connection header received
0808 14:31:21.977293 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:21.977299 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:21.977504 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:21.977657 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:31:21.978068 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:21.979410 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:21.981064 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:21.981875 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:21.981972 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:21.982135 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:21.985152 (+ 3017us) server_negotiation.cc:299] Negotiation successful
0808 14:31:21.985260 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":71,"threads_started":1}
I20260808 14:31:21.988732 21213 tablet_service.cc:1467] Processing CreateTablet for tablet 319b094c6fef4cf597375a7b8a60b6ce (DEFAULT_TABLE table=test-table [id=5c459b93a60b476f978a272d5f6fb404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:31:21.989585 21607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:21.979786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:37837 (local address 127.0.0.1:38780)
0808 14:31:21.983107 (+ 3321us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:21.983119 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:21.983182 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:21.983834 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:21.983836 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:21.983852 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:31:21.983914 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:21.983919 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:21.986903 (+ 2984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:21.986907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:21.988017 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:21.988027 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:21.989188 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:21.989193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:21.989283 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:21.989310 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 14:31:21.989373 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 14:31:21.989444 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3269,"mutex_wait_us":2,"thread_start_us":89,"threads_started":1}
I20260808 14:31:21.989948 21213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 319b094c6fef4cf597375a7b8a60b6ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:31:21.990151 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:21.980199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38780 (local address 127.7.125.67:37837)
0808 14:31:21.981694 (+ 1495us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:21.981701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:21.983277 (+ 1576us) server_negotiation.cc:407] Connection header received
0808 14:31:21.983360 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:21.983366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:21.983551 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:21.983688 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:31:21.984037 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:21.986784 (+ 2747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:21.988178 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:21.988972 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:21.989071 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:21.989840 (+ 769us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:21.989896 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 14:31:21.990001 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1390,"thread_start_us":110,"threads_started":1}
I20260808 14:31:21.990585 21609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:21.978623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:40391 (local address 127.0.0.1:47586)
0808 14:31:21.981847 (+ 3224us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:21.981863 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:21.981959 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:21.983087 (+ 1128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:21.983093 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:21.983112 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:31:21.983213 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:21.983222 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:21.984829 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:21.984832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:21.987472 (+ 2640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:21.987482 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:21.988808 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:21.988811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:21.988895 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:21.988920 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 14:31:21.990394 (+ 1474us) client_negotiation.cc:240] Negotiation successful
0808 14:31:21.990468 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3152,"mutex_wait_us":165,"thread_start_us":115,"threads_started":1}
I20260808 14:31:21.991241 21610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:21.979057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47586 (local address 127.7.125.66:40391)
0808 14:31:21.982196 (+ 3139us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:21.982203 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:21.982226 (+ 23us) server_negotiation.cc:407] Connection header received
0808 14:31:21.982312 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:21.982317 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:21.982528 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:21.982664 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:31:21.983362 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:21.984704 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:21.987601 (+ 2897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:21.988489 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:21.988585 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:21.988742 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:21.991016 (+ 2274us) server_negotiation.cc:299] Negotiation successful
0808 14:31:21.991139 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3030,"thread_start_us":90,"threads_started":1}
I20260808 14:31:21.993155 21503 tablet_service.cc:1467] Processing CreateTablet for tablet 319b094c6fef4cf597375a7b8a60b6ce (DEFAULT_TABLE table=test-table [id=5c459b93a60b476f978a272d5f6fb404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:31:21.993845 21361 tablet_service.cc:1467] Processing CreateTablet for tablet 319b094c6fef4cf597375a7b8a60b6ce (DEFAULT_TABLE table=test-table [id=5c459b93a60b476f978a272d5f6fb404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:31:21.994266 21503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 319b094c6fef4cf597375a7b8a60b6ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:31:21.994822 21361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 319b094c6fef4cf597375a7b8a60b6ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:31:22.005095 21613 tablet_bootstrap.cc:492] T 319b094c6fef4cf597375a7b8a60b6ce P c6ebcbc987fe4fc5ac7c214938bf1a40: Bootstrap starting.
I20260808 14:31:22.007615 21613 tablet_bootstrap.cc:654] T 319b094c6fef4cf597375a7b8a60b6ce P c6ebcbc987fe4fc5ac7c214938bf1a40: Neither blocks nor log segments found. Creating new log.
I20260808 14:31:22.008643 21613 log.cc:824] T 319b094c6fef4cf597375a7b8a60b6ce P c6ebcbc987fe4fc5ac7c214938bf1a40: Log is configured to *not* fsync() on all Append() calls
I20260808 14:31:22.011036 21615 tablet_bootstrap.cc:492] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164: Bootstrap starting.
I20260808 14:31:22.013541 21613 tablet_bootstrap.cc:492] T 319b094c6fef4cf597375a7b8a60b6ce P c6ebcbc987fe4fc5ac7c214938bf1a40: No bootstrap required, opened a new log
I20260808 14:31:22.013773 21613 ts_tablet_manager.cc:1397] T 319b094c6fef4cf597375a7b8a60b6ce P c6ebcbc987fe4fc5ac7c214938bf1a40: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260808 14:31:22.017108 21615 tablet_bootstrap.cc:654] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164: Neither blocks nor log segments found. Creating new log.
I20260808 14:31:22.018000 21615 log.cc:824] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164: Log is configured to *not* fsync() on all Append() calls
I20260808 14:31:22.018141 21613 raft_consensus.cc:348] T 319b094c6fef4cf597375a7b8a60b6ce P c6ebcbc987fe4fc5ac7c214938bf1a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc88cbf66de2433182856da91d317164" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35399 } } peers { permanent_uuid: "0d0be99a4c7848a3828ee93f614f7cdb" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 40391 } } peers { permanent_uuid: "c6ebcbc987fe4fc5ac7c214938bf1a40" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37837 } }
I20260808 14:31:22.018456 21613 raft_consensus.cc:374] T 319b094c6fef4cf597375a7b8a60b6ce P c6ebcbc987fe4fc5ac7c214938bf1a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:31:22.018524 21613 raft_consensus.cc:729] T 319b094c6fef4cf597375a7b8a60b6ce P c6ebcbc987fe4fc5ac7c214938bf1a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6ebcbc987fe4fc5ac7c214938bf1a40, State: Initialized, Role: FOLLOWER
I20260808 14:31:22.020306 21613 consensus_queue.cc:260] T 319b094c6fef4cf597375a7b8a60b6ce P c6ebcbc987fe4fc5ac7c214938bf1a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc88cbf66de2433182856da91d317164" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35399 } } peers { permanent_uuid: "0d0be99a4c7848a3828ee93f614f7cdb" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 40391 } } peers { permanent_uuid: "c6ebcbc987fe4fc5ac7c214938bf1a40" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37837 } }
I20260808 14:31:22.021270 21615 tablet_bootstrap.cc:492] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164: No bootstrap required, opened a new log
I20260808 14:31:22.021436 21613 ts_tablet_manager.cc:1428] T 319b094c6fef4cf597375a7b8a60b6ce P c6ebcbc987fe4fc5ac7c214938bf1a40: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260808 14:31:22.021505 21615 ts_tablet_manager.cc:1397] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260808 14:31:22.021804 21576 heartbeater.cc:503] Master 127.7.125.126:44433 was elected leader, sending a full tablet report...
I20260808 14:31:22.026618 21615 raft_consensus.cc:348] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc88cbf66de2433182856da91d317164" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35399 } } peers { permanent_uuid: "0d0be99a4c7848a3828ee93f614f7cdb" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 40391 } } peers { permanent_uuid: "c6ebcbc987fe4fc5ac7c214938bf1a40" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37837 } }
I20260808 14:31:22.026958 21615 raft_consensus.cc:374] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:31:22.027047 21615 raft_consensus.cc:729] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc88cbf66de2433182856da91d317164, State: Initialized, Role: FOLLOWER
I20260808 14:31:22.028167 21615 consensus_queue.cc:260] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc88cbf66de2433182856da91d317164" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35399 } } peers { permanent_uuid: "0d0be99a4c7848a3828ee93f614f7cdb" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 40391 } } peers { permanent_uuid: "c6ebcbc987fe4fc5ac7c214938bf1a40" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37837 } }
I20260808 14:31:22.029258 21615 ts_tablet_manager.cc:1428] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164: Time spent starting tablet: real 0.008s user 0.004s sys 0.003s
I20260808 14:31:22.029623 21286 heartbeater.cc:503] Master 127.7.125.126:44433 was elected leader, sending a full tablet report...
I20260808 14:31:22.040843 21618 tablet_bootstrap.cc:492] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb: Bootstrap starting.
I20260808 14:31:22.043888 21619 raft_consensus.cc:482] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:31:22.044126 21618 tablet_bootstrap.cc:654] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb: Neither blocks nor log segments found. Creating new log.
I20260808 14:31:22.044097 21619 raft_consensus.cc:504] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc88cbf66de2433182856da91d317164" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35399 } } peers { permanent_uuid: "0d0be99a4c7848a3828ee93f614f7cdb" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 40391 } } peers { permanent_uuid: "c6ebcbc987fe4fc5ac7c214938bf1a40" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37837 } }
I20260808 14:31:22.045114 21618 log.cc:824] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb: Log is configured to *not* fsync() on all Append() calls
I20260808 14:31:22.048276 21619 leader_election.cc:288] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d0be99a4c7848a3828ee93f614f7cdb (127.7.125.66:40391), c6ebcbc987fe4fc5ac7c214938bf1a40 (127.7.125.67:37837)
I20260808 14:31:22.054359 21610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:22.045900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:34045 (local address 127.7.125.66:40391)
0808 14:31:22.046012 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:22.046016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:22.048649 (+ 2633us) server_negotiation.cc:407] Connection header received
0808 14:31:22.048870 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:22.048874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:22.048923 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:22.048973 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:31:22.049433 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:22.050199 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:22.052886 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:22.053534 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:22.053638 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:22.053694 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:22.054109 (+ 415us) server_negotiation.cc:299] Negotiation successful
0808 14:31:22.054197 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 14:31:22.054883 21624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:22.045541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:40391 (local address 127.7.125.65:34045)
0808 14:31:22.048545 (+ 3004us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:22.048563 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:22.048741 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:22.049125 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:22.049128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:22.049147 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:31:22.049305 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:22.049313 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:22.050361 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:22.050365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:22.051644 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:22.051657 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:22.053810 (+ 2153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:22.053814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:22.053949 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:22.053976 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 14:31:22.054593 (+ 617us) client_negotiation.cc:240] Negotiation successful
0808 14:31:22.054697 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2929,"mutex_wait_us":15,"thread_start_us":122,"threads_started":1}
I20260808 14:31:22.056691 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:22.048142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:44715 (local address 127.7.125.67:37837)
0808 14:31:22.051111 (+ 2969us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:22.051115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:22.051990 (+ 875us) server_negotiation.cc:407] Connection header received
0808 14:31:22.052187 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:22.052190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:22.052242 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:22.052310 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:31:22.052759 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:22.053523 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:22.055419 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:22.056000 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:22.056089 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:22.056134 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:22.056501 (+ 367us) server_negotiation.cc:299] Negotiation successful
0808 14:31:22.056581 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2870}
I20260808 14:31:22.057174 21623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:22.046333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:37837 (local address 127.7.125.65:44715)
0808 14:31:22.051896 (+ 5563us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:22.051911 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:22.052063 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:22.052455 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:22.052458 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:22.052476 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:31:22.052613 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:22.052619 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:22.053666 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:22.053670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:22.055261 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:22.055273 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:22.056256 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:22.056259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:22.056363 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:22.056383 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 14:31:22.056972 (+ 589us) client_negotiation.cc:240] Negotiation successful
0808 14:31:22.057048 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5496,"thread_start_us":123,"threads_started":1}
I20260808 14:31:22.055902 21382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "319b094c6fef4cf597375a7b8a60b6ce" candidate_uuid: "cc88cbf66de2433182856da91d317164" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d0be99a4c7848a3828ee93f614f7cdb" is_pre_election: true
W20260808 14:31:22.058243 21168 leader_election.cc:341] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0d0be99a4c7848a3828ee93f614f7cdb (127.7.125.66:40391): Illegal state: must be running to vote when last-logged opid is not known
I20260808 14:31:22.058854 21510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "319b094c6fef4cf597375a7b8a60b6ce" candidate_uuid: "cc88cbf66de2433182856da91d317164" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6ebcbc987fe4fc5ac7c214938bf1a40" is_pre_election: true
I20260808 14:31:22.059228 21510 raft_consensus.cc:2454] T 319b094c6fef4cf597375a7b8a60b6ce P c6ebcbc987fe4fc5ac7c214938bf1a40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc88cbf66de2433182856da91d317164 in term 0.
I20260808 14:31:22.059769 21167 leader_election.cc:302] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c6ebcbc987fe4fc5ac7c214938bf1a40, cc88cbf66de2433182856da91d317164; no voters: 0d0be99a4c7848a3828ee93f614f7cdb
I20260808 14:31:22.060283 21620 raft_consensus.cc:2781] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:31:22.060392 21620 raft_consensus.cc:482] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:31:22.060441 21620 raft_consensus.cc:3037] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:31:22.061894 21620 raft_consensus.cc:504] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc88cbf66de2433182856da91d317164" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35399 } } peers { permanent_uuid: "0d0be99a4c7848a3828ee93f614f7cdb" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 40391 } } peers { permanent_uuid: "c6ebcbc987fe4fc5ac7c214938bf1a40" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37837 } }
I20260808 14:31:22.062708 21510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "319b094c6fef4cf597375a7b8a60b6ce" candidate_uuid: "cc88cbf66de2433182856da91d317164" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6ebcbc987fe4fc5ac7c214938bf1a40"
I20260808 14:31:22.062860 21510 raft_consensus.cc:3037] T 319b094c6fef4cf597375a7b8a60b6ce P c6ebcbc987fe4fc5ac7c214938bf1a40 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:31:22.064420 21510 raft_consensus.cc:2454] T 319b094c6fef4cf597375a7b8a60b6ce P c6ebcbc987fe4fc5ac7c214938bf1a40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc88cbf66de2433182856da91d317164 in term 1.
I20260808 14:31:22.064755 21167 leader_election.cc:302] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6ebcbc987fe4fc5ac7c214938bf1a40, cc88cbf66de2433182856da91d317164; no voters:
I20260808 14:31:22.064849 21382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "319b094c6fef4cf597375a7b8a60b6ce" candidate_uuid: "cc88cbf66de2433182856da91d317164" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d0be99a4c7848a3828ee93f614f7cdb"
I20260808 14:31:22.065147 21619 raft_consensus.cc:2781] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164 [term 1 FOLLOWER]: Leader election won for term 1
W20260808 14:31:22.065413 21287 tablet.cc:2369] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:31:22.065420 21619 raft_consensus.cc:686] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164 [term 1 LEADER]: Becoming Leader. State: Replica: cc88cbf66de2433182856da91d317164, State: Running, Role: LEADER
I20260808 14:31:22.065850 21619 consensus_queue.cc:237] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc88cbf66de2433182856da91d317164" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35399 } } peers { permanent_uuid: "0d0be99a4c7848a3828ee93f614f7cdb" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 40391 } } peers { permanent_uuid: "c6ebcbc987fe4fc5ac7c214938bf1a40" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37837 } }
W20260808 14:31:22.066112 21168 leader_election.cc:341] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0d0be99a4c7848a3828ee93f614f7cdb (127.7.125.66:40391): Illegal state: must be running to vote when last-logged opid is not known
I20260808 14:31:22.068064 21620 leader_election.cc:288] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164 [CANDIDATE]: Term 1 election: Requested vote from peers 0d0be99a4c7848a3828ee93f614f7cdb (127.7.125.66:40391), c6ebcbc987fe4fc5ac7c214938bf1a40 (127.7.125.67:37837)
I20260808 14:31:22.069721 21078 catalog_manager.cc:5495] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc88cbf66de2433182856da91d317164 (127.7.125.65). New cstate: current_term: 1 leader_uuid: "cc88cbf66de2433182856da91d317164" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc88cbf66de2433182856da91d317164" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d0be99a4c7848a3828ee93f614f7cdb" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 40391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6ebcbc987fe4fc5ac7c214938bf1a40" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37837 } health_report { overall_health: UNKNOWN } } }
I20260808 14:31:22.074424 21618 tablet_bootstrap.cc:492] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb: No bootstrap required, opened a new log
I20260808 14:31:22.074676 21618 ts_tablet_manager.cc:1397] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb: Time spent bootstrapping tablet: real 0.034s user 0.005s sys 0.000s
I20260808 14:31:22.077972 21618 raft_consensus.cc:348] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc88cbf66de2433182856da91d317164" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35399 } } peers { permanent_uuid: "0d0be99a4c7848a3828ee93f614f7cdb" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 40391 } } peers { permanent_uuid: "c6ebcbc987fe4fc5ac7c214938bf1a40" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37837 } }
I20260808 14:31:22.078259 21618 raft_consensus.cc:374] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:31:22.078329 21618 raft_consensus.cc:729] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d0be99a4c7848a3828ee93f614f7cdb, State: Initialized, Role: FOLLOWER
I20260808 14:31:22.078929 21618 consensus_queue.cc:260] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc88cbf66de2433182856da91d317164" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35399 } } peers { permanent_uuid: "0d0be99a4c7848a3828ee93f614f7cdb" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 40391 } } peers { permanent_uuid: "c6ebcbc987fe4fc5ac7c214938bf1a40" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37837 } }
I20260808 14:31:22.080163 21618 ts_tablet_manager.cc:1428] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 14:31:22.080540 21432 heartbeater.cc:503] Master 127.7.125.126:44433 was elected leader, sending a full tablet report...
I20260808 14:31:22.089903 21608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:22.085023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59900 (local address 127.7.125.65:35399)
0808 14:31:22.085138 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:22.085142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:22.085204 (+ 62us) server_negotiation.cc:407] Connection header received
0808 14:31:22.085414 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:22.085418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:22.085482 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:22.085590 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:31:22.085845 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:22.086361 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:22.087363 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:22.087690 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:22.087794 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:22.088230 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:31:22.088805 (+ 575us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:31:22.088928 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:22.089685 (+ 757us) server_negotiation.cc:299] Negotiation successful
0808 14:31:22.089750 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 14:31:22.095050 21627 raft_consensus.cc:482] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:31:22.095268 21627 raft_consensus.cc:504] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc88cbf66de2433182856da91d317164" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 35399 } } peers { permanent_uuid: "0d0be99a4c7848a3828ee93f614f7cdb" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 40391 } } peers { permanent_uuid: "c6ebcbc987fe4fc5ac7c214938bf1a40" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37837 } }
I20260808 14:31:22.096537 21627 leader_election.cc:288] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc88cbf66de2433182856da91d317164 (127.7.125.65:35399), c6ebcbc987fe4fc5ac7c214938bf1a40 (127.7.125.67:37837)
I20260808 14:31:22.098690 21510 raft_consensus.cc:1261] T 319b094c6fef4cf597375a7b8a60b6ce P c6ebcbc987fe4fc5ac7c214938bf1a40 [term 1 FOLLOWER]: Refusing update from remote peer cc88cbf66de2433182856da91d317164: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:31:22.099766 21625 consensus_queue.cc:1035] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6ebcbc987fe4fc5ac7c214938bf1a40" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 37837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:31:22.105726 21382 raft_consensus.cc:3037] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:31:22.107475 21382 raft_consensus.cc:1261] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb [term 1 FOLLOWER]: Refusing update from remote peer cc88cbf66de2433182856da91d317164: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:31:22.108304 21625 consensus_queue.cc:1035] T 319b094c6fef4cf597375a7b8a60b6ce P cc88cbf66de2433182856da91d317164 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d0be99a4c7848a3828ee93f614f7cdb" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 40391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 14:31:22.123517 21433 tablet.cc:2369] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:31:22.124956 21629 mvcc.cc:205] Tried to move back new op lower bound from 7316273078665216000 to 7316273078545960960. Current Snapshot: MvccSnapshot[applied={T|T < 7316273078665216000}]
I20260808 14:31:22.130483 21635 mvcc.cc:205] Tried to move back new op lower bound from 7316273078665216000 to 7316273078545960960. Current Snapshot: MvccSnapshot[applied={T|T < 7316273078665216000}]
I20260808 14:31:22.133427 21632 mvcc.cc:205] Tried to move back new op lower bound from 7316273078665216000 to 7316273078545960960. Current Snapshot: MvccSnapshot[applied={T|T < 7316273078665216000}]
I20260808 14:31:22.137981 21608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:22.097794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:45813 (local address 127.7.125.65:35399)
0808 14:31:22.098086 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:22.098090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:22.122550 (+ 24460us) server_negotiation.cc:407] Connection header received
0808 14:31:22.122928 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:22.122931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:22.123008 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:22.123055 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:31:22.130951 (+ 7896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:22.131780 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:22.136700 (+ 4920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:22.137232 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:22.137344 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:22.137415 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:22.137768 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 14:31:22.137860 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219}
I20260808 14:31:22.138461 21631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:22.096927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:35399 (local address 127.7.125.66:45813)
0808 14:31:22.122447 (+ 25520us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:22.122466 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:22.122800 (+ 334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:22.123188 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:22.123191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:22.123208 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:31:22.130801 (+ 7593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:22.130811 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:22.131941 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:22.131944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:22.133212 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:22.133224 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:22.137523 (+ 4299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:22.137527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:22.137625 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:22.137647 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 14:31:22.138183 (+ 536us) client_negotiation.cc:240] Negotiation successful
0808 14:31:22.138270 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25433,"thread_start_us":168,"threads_started":1}
I20260808 14:31:22.139015 21237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "319b094c6fef4cf597375a7b8a60b6ce" candidate_uuid: "0d0be99a4c7848a3828ee93f614f7cdb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc88cbf66de2433182856da91d317164" is_pre_election: true
I20260808 14:31:22.141811 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:22.097858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:59603 (local address 127.7.125.67:37837)
0808 14:31:22.097988 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:22.097993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:22.133868 (+ 35875us) server_negotiation.cc:407] Connection header received
0808 14:31:22.134055 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:22.134058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:22.134105 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:22.134151 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:31:22.134539 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:22.135266 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:22.140494 (+ 5228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:22.141076 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:22.141180 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:22.141228 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:22.141613 (+ 385us) server_negotiation.cc:299] Negotiation successful
0808 14:31:22.141692 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:31:22.143057 21630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:22.097390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:37837 (local address 127.7.125.66:59603)
0808 14:31:22.133769 (+ 36379us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:22.133785 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:22.133941 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:22.134275 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:22.134278 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:22.134305 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:31:22.134416 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:22.134423 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:22.135439 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:22.135443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:22.136536 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:22.136550 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:22.141339 (+ 4789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:22.141342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:22.141448 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:22.141470 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 14:31:22.142785 (+ 1315us) client_negotiation.cc:240] Negotiation successful
0808 14:31:22.142874 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36296,"thread_start_us":116,"threads_started":1}
I20260808 14:31:22.143543 21510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "319b094c6fef4cf597375a7b8a60b6ce" candidate_uuid: "0d0be99a4c7848a3828ee93f614f7cdb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6ebcbc987fe4fc5ac7c214938bf1a40" is_pre_election: true
I20260808 14:31:22.144104 21309 leader_election.cc:302] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d0be99a4c7848a3828ee93f614f7cdb; no voters: c6ebcbc987fe4fc5ac7c214938bf1a40, cc88cbf66de2433182856da91d317164
I20260808 14:31:22.144661 21627 raft_consensus.cc:2726] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 14:31:22.145354 21072 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:31:22.145519 21072 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:31:22.148607 21072 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 426aefb2448b43c780ef838b429197dd: Sending DeleteTablet for 3 replicas of tablet 319b094c6fef4cf597375a7b8a60b6ce
I20260808 14:31:22.149688 21361 tablet_service.cc:1514] Processing DeleteTablet for tablet 319b094c6fef4cf597375a7b8a60b6ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:31:22 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:47586
I20260808 14:31:22.151171 21213 tablet_service.cc:1514] Processing DeleteTablet for tablet 319b094c6fef4cf597375a7b8a60b6ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:31:22 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:59898
I20260808 14:31:22.151722 21503 tablet_service.cc:1514] Processing DeleteTablet for tablet 319b094c6fef4cf597375a7b8a60b6ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:31:22 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:38780
I20260808 14:31:22.153052 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 21119
I20260808 14:31:22.153976 21641 tablet_replica.cc:330] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb: stopping tablet replica
I20260808 14:31:22.154337 21641 raft_consensus.cc:2229] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:31:22.154693 21641 raft_consensus.cc:2258] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:31:22.156448 21641 ts_tablet_manager.cc:1905] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:31:22.158558 21643 tablet_replica.cc:330] T 319b094c6fef4cf597375a7b8a60b6ce P c6ebcbc987fe4fc5ac7c214938bf1a40: stopping tablet replica
I20260808 14:31:22.159727 21641 ts_tablet_manager.cc:1918] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:31:22.159842 21641 log.cc:1196] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/319b094c6fef4cf597375a7b8a60b6ce
I20260808 14:31:22.160209 21641 ts_tablet_manager.cc:1939] T 319b094c6fef4cf597375a7b8a60b6ce P 0d0be99a4c7848a3828ee93f614f7cdb: Deleting consensus metadata
I20260808 14:31:22.160403 21643 raft_consensus.cc:2229] T 319b094c6fef4cf597375a7b8a60b6ce P c6ebcbc987fe4fc5ac7c214938bf1a40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:31:22.160866 21643 raft_consensus.cc:2258] T 319b094c6fef4cf597375a7b8a60b6ce P c6ebcbc987fe4fc5ac7c214938bf1a40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:31:22.161669 21062 catalog_manager.cc:4841] TS 0d0be99a4c7848a3828ee93f614f7cdb (127.7.125.66:40391): tablet 319b094c6fef4cf597375a7b8a60b6ce (table test-table [id=5c459b93a60b476f978a272d5f6fb404]) successfully deleted
I20260808 14:31:22.162849 21643 ts_tablet_manager.cc:1905] T 319b094c6fef4cf597375a7b8a60b6ce P c6ebcbc987fe4fc5ac7c214938bf1a40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:31:22.165766 21643 ts_tablet_manager.cc:1918] T 319b094c6fef4cf597375a7b8a60b6ce P c6ebcbc987fe4fc5ac7c214938bf1a40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:31:22.165884 21643 log.cc:1196] T 319b094c6fef4cf597375a7b8a60b6ce P c6ebcbc987fe4fc5ac7c214938bf1a40: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/319b094c6fef4cf597375a7b8a60b6ce
I20260808 14:31:22.166232 21643 ts_tablet_manager.cc:1939] T 319b094c6fef4cf597375a7b8a60b6ce P c6ebcbc987fe4fc5ac7c214938bf1a40: Deleting consensus metadata
I20260808 14:31:22.168025 21061 catalog_manager.cc:4841] TS c6ebcbc987fe4fc5ac7c214938bf1a40 (127.7.125.67:37837): tablet 319b094c6fef4cf597375a7b8a60b6ce (table test-table [id=5c459b93a60b476f978a272d5f6fb404]) successfully deleted
I20260808 14:31:22.168319 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 21290
W20260808 14:31:22.168350 21061 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.125.65:35399 (error 108)
I20260808 14:31:22.171969 21609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:22.171550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:35399 (local address 127.0.0.1:59908)
0808 14:31:22.171770 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:22.171853 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.65:35399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260808 14:31:22.172163 21061 catalog_manager.cc:4584] TS cc88cbf66de2433182856da91d317164 (127.7.125.65:35399): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 319b094c6fef4cf597375a7b8a60b6ce: Network error: Client connection negotiation failed: client connection to 127.7.125.65:35399: connect: Connection refused (error 111)
I20260808 14:31:22.179733 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 21435
I20260808 14:31:22.189020 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 21043
I20260808 14:31:22.199486 7669 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 14:31:22.545518 7669 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 14:31:22.545639 7669 mini_ranger.cc:66] Stopping Ranger...
I20260808 14:31:22.890554 7669 mini_ranger.cc:68] Stopped Ranger
2026-08-08 14:31:22.890 UTC [20052] LOG: received smart shutdown request
2026-08-08 14:31:22.895 UTC [20052] LOG: background worker "logical replication launcher" (PID 20064) exited with exit code 1
2026-08-08 14:31:22.896 UTC [20056] LOG: shutting down
2026-08-08 14:31:23.092 UTC [20052] LOG: database system is shut down
2026-08-08T14:31:23Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50597 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:31:23 dist-test-slave-w206 krb5kdc[21653](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:31:23 dist-test-slave-w206 krb5kdc[21653](info): set up 2 sockets
Aug 08 14:31:23 dist-test-slave-w206 krb5kdc[21653](info): commencing operation
krb5kdc: starting...
W20260808 14:31:24.335901 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:31:24 dist-test-slave-w206 krb5kdc[21653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:31:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:31:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:31:24.469693 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:33863
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:35305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:33863,127.7.125.125:37525,127.7.125.124:43331
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:31:24.585165 21666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:24.585438 21666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:24.585477 21666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:24.589478 21666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:31:24.589534 21666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:24.589555 21666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:24.589571 21666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:31:24.589586 21666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:31:24.594870 21666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:35305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:33863,127.7.125.125:37525,127.7.125.124:43331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:33863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:24.595948 21666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:24.596939 21666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:24.605969 21671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:24.605971 21673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:24.606091 21675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:24.606220 21666 server_base.cc:1034] running on GCE node
I20260808 14:31:24.606781 21666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:24.607905 21666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:24.609069 21666 hybrid_clock.cc:648] HybridClock initialized: now 1786199484609039 us; error 42 us; skew 500 ppm
Aug 08 14:31:24 dist-test-slave-w206 krb5kdc[21653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199484, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:24.611999 21666 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:31:24.612926 21666 webserver.cc:460] Webserver started at http://127.7.125.126:34183/ using document root <none> and password file <none>
I20260808 14:31:24.613482 21666 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:24.613569 21666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:24.613809 21666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:31:24.615949 21666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "991bb3d755524e53b49d34c65bbc3bc9"
format_stamp: "Formatted at 2026-08-08 14:31:24 on dist-test-slave-w206"
server_key: "45e7c0bd9259c8f256362309c602228e"
server_key_iv: "83453ca221cb2421ca18c0ce4d72e6bf"
server_key_version: "encryptionkey@0"
I20260808 14:31:24.616503 21666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "991bb3d755524e53b49d34c65bbc3bc9"
format_stamp: "Formatted at 2026-08-08 14:31:24 on dist-test-slave-w206"
server_key: "45e7c0bd9259c8f256362309c602228e"
server_key_iv: "83453ca221cb2421ca18c0ce4d72e6bf"
server_key_version: "encryptionkey@0"
I20260808 14:31:24.620213 21666 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:31:24.622611 21683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:24.623908 21666 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:31:24.624068 21666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "991bb3d755524e53b49d34c65bbc3bc9"
format_stamp: "Formatted at 2026-08-08 14:31:24 on dist-test-slave-w206"
server_key: "45e7c0bd9259c8f256362309c602228e"
server_key_iv: "83453ca221cb2421ca18c0ce4d72e6bf"
server_key_version: "encryptionkey@0"
I20260808 14:31:24.624188 21666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:24.635903 21666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:24.639429 21666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:24.639643 21666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:24.647188 21666 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:33863
I20260808 14:31:24.647213 21738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:33863 every 8 connection(s)
I20260808 14:31:24.648162 21666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:31:24.650959 21739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:31:24.654778 21739 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33863 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:31:24.656999 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 21666
I20260808 14:31:24.657162 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:31:24.657467 7669 external_mini_cluster.cc:1428] Setting key 6fcdea97b873e2d87c1c0923ec2808a4
Aug 08 14:31:24 dist-test-slave-w206 krb5kdc[21653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199484, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.125.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.125@KRBTEST.COM" created.
I20260808 14:31:24.672554 21742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:24.657276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:33863 (local address 127.0.0.1:45340)
0808 14:31:24.657909 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:24.657958 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:24.658846 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:24.659790 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:24.659799 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:24.660407 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:24.660822 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:24.660844 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:24.662595 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:24.662598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:24.663415 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:24.663426 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:24.664436 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:24.664440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:24.664572 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:24.665246 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:24.665271 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:24.667096 (+ 1825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:24.669175 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:24.669193 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:24.669203 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:24.669433 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:24.669848 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:24.669853 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:24.669855 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:24.669979 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:24.670741 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:24.670748 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:24.670897 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 14:31:24.671095 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 14:31:24.671493 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":107,"spinlock_wait_cycles":54528,"thread_start_us":156,"threads_started":1}
I20260808 14:31:24.672559 21741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:24.657276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45340 (local address 127.7.125.126:33863)
0808 14:31:24.657806 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:24.657827 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:24.658081 (+ 254us) server_negotiation.cc:407] Connection header received
0808 14:31:24.659239 (+ 1158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:24.659248 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:24.659503 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:24.659737 (+ 234us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:24.661000 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:24.662449 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:24.663551 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:24.664254 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:24.664457 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:24.667272 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:24.667278 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:24.667288 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:24.667310 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:24.669060 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:24.669561 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:24.669566 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:24.669567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:24.669743 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:24.670086 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:24.670089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:24.670090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:24.670620 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:24.670773 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:24.671177 (+ 404us) server_negotiation.cc:299] Negotiation successful
0808 14:31:24.671440 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":358,"thread_start_us":147,"threads_started":1}
I20260808 14:31:24.675096 21739 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:31:24.675922 21742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:24.675642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:37525 (local address 127.0.0.1:58786)
0808 14:31:24.675774 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:24.675858 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.125:37525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 14:31:24.676254 21685 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.125.125:37525: connect: Connection refused (error 111)
I20260808 14:31:24.677217 21742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:24.676994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:37525 (local address 127.0.0.1:58796)
0808 14:31:24.677107 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:24.677152 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.125:37525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 14:31:24.677446 21739 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.125:37525: Network error: Client connection negotiation failed: client connection to 127.7.125.125:37525: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.125.keytab.
Entry for principal kudu/127.7.125.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.125.keytab.
I20260808 14:31:24.688172 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.125:37525
--webserver_interface=127.7.125.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.125.keytab
--principal=kudu/127.7.125.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:35305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:33863,127.7.125.125:37525,127.7.125.124:43331
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:31:24.726747 21739 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } attempt: 1
I20260808 14:31:24.727778 21742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:24.727457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:37525 (local address 127.0.0.1:58810)
0808 14:31:24.727625 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:24.727691 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.125:37525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 14:31:24.728538 21742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:24.728329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:37525 (local address 127.0.0.1:58814)
0808 14:31:24.728441 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:24.728486 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.125:37525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 14:31:24.728725 21739 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.125:37525: Network error: Client connection negotiation failed: client connection to 127.7.125.125:37525: connect: Connection refused (error 111)
I20260808 14:31:24.796932 21739 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } attempt: 2
I20260808 14:31:24.797907 21742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:24.797547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:37525 (local address 127.0.0.1:58820)
0808 14:31:24.797727 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:24.797800 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.125:37525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260808 14:31:24.798775 21742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:24.798508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:37525 (local address 127.0.0.1:58832)
0808 14:31:24.798627 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:24.798684 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.125:37525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 14:31:24.798919 21739 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.125:37525: Network error: Client connection negotiation failed: client connection to 127.7.125.125:37525: connect: Connection refused (error 111)
W20260808 14:31:24.804047 21746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:24.804354 21746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:24.804431 21746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:24.808574 21746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:31:24.808661 21746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:24.808698 21746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:24.808734 21746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:31:24.808768 21746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:31:24.814419 21746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:35305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:33863,127.7.125.125:37525,127.7.125.124:43331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.125.keytab
--principal=kudu/127.7.125.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.125:37525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.125.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:24.815698 21746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:24.816771 21746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:24.825572 21754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:24.825776 21746 server_base.cc:1034] running on GCE node
W20260808 14:31:24.825666 21752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:24.825572 21751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:24.826501 21746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:24.827755 21746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:24.828994 21746 hybrid_clock.cc:648] HybridClock initialized: now 1786199484828946 us; error 50 us; skew 500 ppm
Aug 08 14:31:24 dist-test-slave-w206 krb5kdc[21653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199484, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:24.832366 21746 init.cc:377] Logged in from keytab as kudu/127.7.125.125@KRBTEST.COM (short username kudu)
I20260808 14:31:24.833305 21746 webserver.cc:460] Webserver started at http://127.7.125.125:33361/ using document root <none> and password file <none>
I20260808 14:31:24.833896 21746 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:24.833993 21746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:24.834239 21746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:31:24.836395 21746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-1/data/instance:
uuid: "c71811e534b5473ea654347fb81af3bd"
format_stamp: "Formatted at 2026-08-08 14:31:24 on dist-test-slave-w206"
server_key: "5f4ba6642e2b19e2e3f9acddfdbf0735"
server_key_iv: "73f2c87fbc0dae1a3b1b7cc71a110a73"
server_key_version: "encryptionkey@0"
I20260808 14:31:24.836977 21746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-1/wal/instance:
uuid: "c71811e534b5473ea654347fb81af3bd"
format_stamp: "Formatted at 2026-08-08 14:31:24 on dist-test-slave-w206"
server_key: "5f4ba6642e2b19e2e3f9acddfdbf0735"
server_key_iv: "73f2c87fbc0dae1a3b1b7cc71a110a73"
server_key_version: "encryptionkey@0"
I20260808 14:31:24.841199 21746 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:31:24.843905 21761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:24.845283 21746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:31:24.845460 21746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-1/wal
uuid: "c71811e534b5473ea654347fb81af3bd"
format_stamp: "Formatted at 2026-08-08 14:31:24 on dist-test-slave-w206"
server_key: "5f4ba6642e2b19e2e3f9acddfdbf0735"
server_key_iv: "73f2c87fbc0dae1a3b1b7cc71a110a73"
server_key_version: "encryptionkey@0"
I20260808 14:31:24.845587 21746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:24.890115 21739 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } attempt: 3
I20260808 14:31:24.891129 21742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:24.890758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:37525 (local address 127.0.0.1:58838)
0808 14:31:24.890936 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:24.891031 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.125:37525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 14:31:24.892011 21742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:24.891744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:37525 (local address 127.0.0.1:58848)
0808 14:31:24.891893 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:24.891954 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.125:37525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260808 14:31:24.892215 21739 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.125:37525: Network error: Client connection negotiation failed: client connection to 127.7.125.125:37525: connect: Connection refused (error 111)
I20260808 14:31:24.894366 21746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:24.897569 21746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:24.897724 21746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:24.905311 21746 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.125:37525
I20260808 14:31:24.905334 21817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.125:37525 every 8 connection(s)
I20260808 14:31:24.906396 21746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-1/data/info.pb
I20260808 14:31:24.909263 21818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:31:24.913162 21818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33863 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:31:24.916615 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 21746
I20260808 14:31:24.916759 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-1/wal/instance
I20260808 14:31:24.917114 7669 external_mini_cluster.cc:1428] Setting key 75618c4e040133c8c9d386f7d7952d1f
Aug 08 14:31:24 dist-test-slave-w206 krb5kdc[21653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199484, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.125@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.125.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.124@KRBTEST.COM" created.
I20260808 14:31:24.929831 21741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:24.915238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45344 (local address 127.7.125.126:33863)
0808 14:31:24.915423 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:24.915426 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:24.916314 (+ 888us) server_negotiation.cc:407] Connection header received
0808 14:31:24.917607 (+ 1293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:24.917610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:24.917665 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:24.917744 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:24.919516 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:24.920412 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:24.921599 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:24.922057 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:24.922194 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:24.925159 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:24.925165 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:24.925171 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:24.925212 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:24.927208 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:24.927901 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:24.927905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:24.927906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:24.928005 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:24.928424 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:24.928427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:24.928429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:24.928675 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:24.928755 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:24.929499 (+ 744us) server_negotiation.cc:299] Negotiation successful
0808 14:31:24.929621 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260808 14:31:24.930943 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:24.915652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:33863 (local address 127.0.0.1:45344)
0808 14:31:24.916136 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:24.916184 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:24.917354 (+ 1170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:24.917955 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:24.917967 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:24.918519 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:24.919314 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:24.919342 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:24.920568 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:24.920570 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:24.921460 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:24.921469 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:24.922248 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:24.922254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:24.922490 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:24.923169 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:24.923187 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:24.924953 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:24.927402 (+ 2449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:24.927411 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:24.927432 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:24.927755 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:24.928106 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:24.928113 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:24.928115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:24.928303 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:24.928780 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:24.928786 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:24.929176 (+ 390us) client_negotiation.cc:769] Sending connection context
0808 14:31:24.929405 (+ 229us) client_negotiation.cc:240] Negotiation successful
0808 14:31:24.929777 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":117,"threads_started":1}
I20260808 14:31:24.932500 21818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 14:31:24 dist-test-slave-w206 krb5kdc[21653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199484, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.125@KRBTEST.COM for kudu/127.7.125.125@KRBTEST.COM
I20260808 14:31:24.942312 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:24.933098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:37525 (local address 127.0.0.1:58856)
0808 14:31:24.933294 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:24.933310 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:24.933397 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:24.933808 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:24.933810 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:24.933953 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:24.934078 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:24.934084 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:24.935160 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:24.935164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:24.935808 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:24.935818 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:24.936444 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:24.936447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:24.936539 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:24.937070 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:24.937092 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:24.938584 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:24.940737 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:24.940739 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:24.940741 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:24.940933 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:24.941222 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:24.941224 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:24.941225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:24.941308 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:24.941855 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:24.941857 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:24.941940 (+ 83us) client_negotiation.cc:769] Sending connection context
0808 14:31:24.942027 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 14:31:24.942135 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115}
I20260808 14:31:24.942441 21823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:24.933178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58856 (local address 127.7.125.125:37525)
0808 14:31:24.933477 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:24.933487 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:24.933506 (+ 19us) server_negotiation.cc:407] Connection header received
0808 14:31:24.933541 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:24.933545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:24.933691 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:24.933798 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:24.934187 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:24.935029 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:24.935926 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:24.936324 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:24.936441 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:24.938751 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:24.938765 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:24.938785 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:24.938816 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:24.940629 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:24.941038 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:24.941042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:24.941044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:24.941124 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:24.941389 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:24.941391 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:24.941392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:24.941733 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:24.941882 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:24.942134 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 14:31:24.942262 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":103,"threads_started":1}
I20260808 14:31:24.944334 21818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:31:24.945127 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:24.944877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:43331 (local address 127.0.0.1:47766)
0808 14:31:24.944997 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:24.945069 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:43331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 14:31:24.945390 21765 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.125.124:43331: connect: Connection refused (error 111)
I20260808 14:31:24.946395 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:24.946119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:43331 (local address 127.0.0.1:47780)
0808 14:31:24.946248 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:24.946304 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:43331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 14:31:24.946638 21818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.124:43331: Network error: Client connection negotiation failed: client connection to 127.7.125.124:43331: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.124.keytab.
Entry for principal kudu/127.7.125.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.124.keytab.
I20260808 14:31:24.950493 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.124:43331
--webserver_interface=127.7.125.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.124.keytab
--principal=kudu/127.7.125.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:35305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:33863,127.7.125.125:37525,127.7.125.124:43331
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:31:24.995886 21818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } attempt: 1
I20260808 14:31:24.996843 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:24.996530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:43331 (local address 127.0.0.1:47788)
0808 14:31:24.996695 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:24.996760 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:43331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 14:31:24.997608 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:24.997374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:43331 (local address 127.0.0.1:47796)
0808 14:31:24.997494 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:24.997549 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:43331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 14:31:24.997746 21818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.124:43331: Network error: Client connection negotiation failed: client connection to 127.7.125.124:43331: connect: Connection refused (error 111)
I20260808 14:31:25.035386 21739 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } attempt: 4
Aug 08 14:31:25 dist-test-slave-w206 krb5kdc[21653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199484, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for kudu/127.7.125.125@KRBTEST.COM
I20260808 14:31:25.044955 21823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.036075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58870 (local address 127.7.125.125:37525)
0808 14:31:25.036263 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:25.036268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:25.036284 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:31:25.036429 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:25.036431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:25.036478 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:25.036563 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:25.037447 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.038307 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.039164 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.039429 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.039544 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:25.041704 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:25.041708 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:25.041710 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:25.041735 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:25.043282 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.043716 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.043720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.043722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.043807 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.044100 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.044105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.044106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.044293 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:25.044409 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:25.044672 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 14:31:25.044792 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260808 14:31:25.044955 21742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.035965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:37525 (local address 127.0.0.1:58870)
0808 14:31:25.036158 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:25.036177 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:25.036288 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:25.036592 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:25.036595 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:25.036793 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:25.037296 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:25.037305 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.038446 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.038450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:25.039036 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:25.039046 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.039561 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.039564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:25.039679 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:25.040244 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:25.040255 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:25.041553 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:25.043395 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:25.043399 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:25.043401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:25.043612 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:25.043909 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:25.043912 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:25.043914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:25.044005 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:25.044392 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:25.044395 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:25.044504 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 14:31:25.044618 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 14:31:25.044772 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260808 14:31:25.045780 21739 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:31:25.046545 21742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.046298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:43331 (local address 127.0.0.1:47804)
0808 14:31:25.046430 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:25.046489 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:43331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 14:31:25.047384 21742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.047188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:43331 (local address 127.0.0.1:47810)
0808 14:31:25.047286 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:25.047329 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:43331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 14:31:25.047597 21739 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.124:43331: Network error: Client connection negotiation failed: client connection to 127.7.125.124:43331: connect: Connection refused (error 111)
I20260808 14:31:25.065917 21818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } attempt: 2
I20260808 14:31:25.066763 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.066469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:43331 (local address 127.0.0.1:47812)
0808 14:31:25.066616 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:25.066687 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:43331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 14:31:25.067579 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.067369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:43331 (local address 127.0.0.1:47816)
0808 14:31:25.067481 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:25.067527 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:43331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 14:31:25.067788 21818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.124:43331: Network error: Client connection negotiation failed: client connection to 127.7.125.124:43331: connect: Connection refused (error 111)
W20260808 14:31:25.069154 21825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:25.069454 21825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:25.069552 21825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:25.073856 21825 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:31:25.073951 21825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:25.074000 21825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:25.074038 21825 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:31:25.074079 21825 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:31:25.079670 21825 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:35305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:33863,127.7.125.125:37525,127.7.125.124:43331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.124.keytab
--principal=kudu/127.7.125.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.124:43331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.125.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:25.080832 21825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:25.081877 21825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:25.090610 21830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:25.090579 21833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:25.090617 21831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:25.090833 21825 server_base.cc:1034] running on GCE node
I20260808 14:31:25.091662 21825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:25.092908 21825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:25.094082 21825 hybrid_clock.cc:648] HybridClock initialized: now 1786199485094062 us; error 33 us; skew 500 ppm
Aug 08 14:31:25 dist-test-slave-w206 krb5kdc[21653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199485, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:25.097174 21825 init.cc:377] Logged in from keytab as kudu/127.7.125.124@KRBTEST.COM (short username kudu)
I20260808 14:31:25.098083 21825 webserver.cc:460] Webserver started at http://127.7.125.124:40885/ using document root <none> and password file <none>
I20260808 14:31:25.098609 21825 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:25.098665 21825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:25.098834 21825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:31:25.101012 21825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-2/data/instance:
uuid: "d2939d45413f4547981b10132e02e2b1"
format_stamp: "Formatted at 2026-08-08 14:31:25 on dist-test-slave-w206"
server_key: "7bcdf42ee0e21ac2f7b91f92012cf891"
server_key_iv: "6670c75cb9f19c965d096434da2029b3"
server_key_version: "encryptionkey@0"
I20260808 14:31:25.101516 21825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-2/wal/instance:
uuid: "d2939d45413f4547981b10132e02e2b1"
format_stamp: "Formatted at 2026-08-08 14:31:25 on dist-test-slave-w206"
server_key: "7bcdf42ee0e21ac2f7b91f92012cf891"
server_key_iv: "6670c75cb9f19c965d096434da2029b3"
server_key_version: "encryptionkey@0"
I20260808 14:31:25.105506 21825 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260808 14:31:25.106776 21739 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } attempt: 1
I20260808 14:31:25.107867 21742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.107549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:43331 (local address 127.0.0.1:47832)
0808 14:31:25.107709 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:25.107779 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:43331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 14:31:25.108350 21840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:25.108764 21742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.108507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:43331 (local address 127.0.0.1:47834)
0808 14:31:25.108632 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:25.108691 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:43331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 14:31:25.108965 21739 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.124:43331: Network error: Client connection negotiation failed: client connection to 127.7.125.124:43331: connect: Connection refused (error 111)
I20260808 14:31:25.109845 21825 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:31:25.110021 21825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-2/wal
uuid: "d2939d45413f4547981b10132e02e2b1"
format_stamp: "Formatted at 2026-08-08 14:31:25 on dist-test-slave-w206"
server_key: "7bcdf42ee0e21ac2f7b91f92012cf891"
server_key_iv: "6670c75cb9f19c965d096434da2029b3"
server_key_version: "encryptionkey@0"
I20260808 14:31:25.110157 21825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:25.122368 21825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:25.125675 21825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:25.125916 21825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:25.134055 21825 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.124:43331
I20260808 14:31:25.134087 21895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.124:43331 every 8 connection(s)
I20260808 14:31:25.135221 21825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-2/data/info.pb
I20260808 14:31:25.138200 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 21825
I20260808 14:31:25.138258 21896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:31:25.138382 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199316863421-7669-0/minicluster-data/master-2/wal/instance
I20260808 14:31:25.138723 7669 external_mini_cluster.cc:1428] Setting key 51e7de04cac830e8dd9335b82b06d2bb
I20260808 14:31:25.143832 21896 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33863 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 14:31:25 dist-test-slave-w206 krb5kdc[21653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.124@KRBTEST.COM
Aug 08 14:31:25 dist-test-slave-w206 krb5kdc[21653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199485, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.124@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:31:25.155309 21899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.140202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47846 (local address 127.7.125.124:43331)
0808 14:31:25.140648 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:25.140670 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:25.140709 (+ 39us) server_negotiation.cc:407] Connection header received
0808 14:31:25.141819 (+ 1110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:25.141841 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:25.142222 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:25.142577 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:25.143470 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.144543 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.145410 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.146053 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.146226 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:25.149277 (+ 3051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:25.149283 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:25.149293 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:25.149352 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:25.151455 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.152020 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.152025 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.152030 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.152139 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.152466 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.152472 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.152473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.153080 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:25.153335 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:25.153816 (+ 481us) server_negotiation.cc:299] Negotiation successful
0808 14:31:25.154165 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":162,"threads_started":1}
I20260808 14:31:25.157130 21741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.145699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45358 (local address 127.7.125.126:33863)
0808 14:31:25.145846 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:25.145851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:25.146205 (+ 354us) server_negotiation.cc:407] Connection header received
0808 14:31:25.146443 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:25.146449 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:25.146513 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:25.146601 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:25.147486 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.148172 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.149357 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.149726 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.149845 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:25.152254 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:25.152260 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:25.152263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:25.152292 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:25.154192 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.154726 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.154730 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.154732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.154832 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.155232 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.155235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.155237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.155498 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:25.156431 (+ 933us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:25.156879 (+ 448us) server_negotiation.cc:299] Negotiation successful
0808 14:31:25.157010 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:31:25.157279 21900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.145621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:33863 (local address 127.0.0.1:45358)
0808 14:31:25.146073 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:25.146126 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:25.146295 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:25.146681 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:25.146688 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:25.147179 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:25.147318 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:25.147324 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.148314 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.148317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:25.149238 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:25.149248 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.149848 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.149852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:25.150068 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:25.150514 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:25.150525 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:25.152103 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:25.154327 (+ 2224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:25.154336 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:25.154341 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:25.154599 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:25.154956 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:25.154958 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:25.154960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:25.155126 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:25.156462 (+ 1336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:25.156470 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:25.156621 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 14:31:25.156816 (+ 195us) client_negotiation.cc:240] Negotiation successful
0808 14:31:25.157051 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":203,"threads_started":1}
I20260808 14:31:25.158711 21896 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:31:25.158962 21818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } attempt: 3
Aug 08 14:31:25 dist-test-slave-w206 krb5kdc[21653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199485, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.124@KRBTEST.COM for kudu/127.7.125.125@KRBTEST.COM
Aug 08 14:31:25 dist-test-slave-w206 krb5kdc[21653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
Aug 08 14:31:25 dist-test-slave-w206 krb5kdc[21653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199484, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.125@KRBTEST.COM for kudu/127.7.125.124@KRBTEST.COM
I20260808 14:31:25.168898 21900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.159360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:37525 (local address 127.0.0.1:58872)
0808 14:31:25.159556 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:25.159574 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:25.159662 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:25.160110 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:25.160113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:25.160318 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:25.160492 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:25.160500 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.161563 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.161567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:25.162175 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:25.162184 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.162878 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.162881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:25.163015 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:25.163453 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:25.163472 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:25.164850 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:25.167055 (+ 2205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:25.167060 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:25.167063 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:25.167325 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:25.167680 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:25.167686 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:25.167688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:25.167790 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:25.168274 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:25.168277 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:25.168410 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 14:31:25.168534 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 14:31:25.168686 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":2}
I20260808 14:31:25.168957 21823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.159661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58872 (local address 127.7.125.125:37525)
0808 14:31:25.159871 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:25.159876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:25.159885 (+ 9us) server_negotiation.cc:407] Connection header received
0808 14:31:25.159929 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:25.159932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:25.159970 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:25.160050 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:25.160642 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.161431 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.162308 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.162616 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.162714 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:25.165003 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:25.165007 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:25.165010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:25.165045 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:25.166871 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.167460 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.167464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.167467 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.167559 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.167891 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.167895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.167896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.168166 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:25.168279 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:25.168614 (+ 335us) server_negotiation.cc:299] Negotiation successful
0808 14:31:25.168787 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":134}
I20260808 14:31:25.169974 21896 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:31:25.170082 21741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.158383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45360 (local address 127.7.125.126:33863)
0808 14:31:25.158528 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:25.158536 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:25.158555 (+ 19us) server_negotiation.cc:407] Connection header received
0808 14:31:25.158647 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:25.158651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:25.158704 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:25.158820 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:25.159769 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.160539 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.161689 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.162016 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.162119 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:25.165693 (+ 3574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:25.165697 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:25.165700 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:25.165728 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:25.167643 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.168211 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.168216 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.168219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.168314 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.168652 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.168655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.168657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.168886 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:25.169143 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:25.169794 (+ 651us) server_negotiation.cc:299] Negotiation successful
0808 14:31:25.169929 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 14:31:25.170554 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.159488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:43331 (local address 127.0.0.1:47852)
0808 14:31:25.159623 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:25.159639 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:25.159730 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:25.160419 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:25.160423 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:25.160638 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:25.160793 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:25.160799 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.161536 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.161541 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:25.162314 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:25.162323 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.162932 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.162936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:25.163046 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:25.163612 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:25.163626 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:25.165964 (+ 2338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:25.168638 (+ 2674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:25.168643 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:25.168648 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:25.168924 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:25.169455 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:25.169458 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:25.169460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:25.169555 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:25.170016 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:25.170020 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:25.170122 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 14:31:25.170235 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 14:31:25.170370 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":37}
I20260808 14:31:25.171203 21899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.159519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47852 (local address 127.7.125.124:43331)
0808 14:31:25.160198 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:25.160202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:25.160213 (+ 11us) server_negotiation.cc:407] Connection header received
0808 14:31:25.160244 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:25.160246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:25.160281 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:25.160357 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:25.160913 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.161423 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.162428 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.162808 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.162929 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:25.166118 (+ 3189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:25.166121 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:25.166124 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:25.166148 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:25.168525 (+ 2377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.169249 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.169252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.169254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.169345 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.169675 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.169679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.169681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.169905 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:25.170012 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:25.170862 (+ 850us) server_negotiation.cc:299] Negotiation successful
0808 14:31:25.171037 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"spinlock_wait_cycles":4480}
I20260808 14:31:25.174825 21818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c71811e534b5473ea654347fb81af3bd: Bootstrap starting.
I20260808 14:31:25.176149 21739 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } attempt: 2
Aug 08 14:31:25 dist-test-slave-w206 krb5kdc[21653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199485, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.124@KRBTEST.COM for kudu/127.7.125.124@KRBTEST.COM
Aug 08 14:31:25 dist-test-slave-w206 krb5kdc[21653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.125@KRBTEST.COM
I20260808 14:31:25.177976 21818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c71811e534b5473ea654347fb81af3bd: Neither blocks nor log segments found. Creating new log.
I20260808 14:31:25.179077 21818 log.cc:824] T 00000000000000000000000000000000 P c71811e534b5473ea654347fb81af3bd: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:31:25 dist-test-slave-w206 krb5kdc[21653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199484, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for kudu/127.7.125.124@KRBTEST.COM
I20260808 14:31:25.181823 21818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c71811e534b5473ea654347fb81af3bd: No bootstrap required, opened a new log
I20260808 14:31:25.182960 21823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.171366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58876 (local address 127.7.125.125:37525)
0808 14:31:25.171482 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:25.171486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:25.171836 (+ 350us) server_negotiation.cc:407] Connection header received
0808 14:31:25.171978 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:25.171981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:25.172025 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:25.172072 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:25.173001 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.173703 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.174638 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.175036 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.175136 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:25.178811 (+ 3675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:25.178815 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:25.178818 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:25.178846 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:25.180689 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.181258 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.181262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.181263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.181353 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.181802 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.181805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.181807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.182057 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:25.182210 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:25.182663 (+ 453us) server_negotiation.cc:299] Negotiation successful
0808 14:31:25.182792 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:31:25.183326 21901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.170706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47858 (local address 127.7.125.124:43331)
0808 14:31:25.171455 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:25.171466 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:25.171481 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:31:25.171512 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:25.171515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:25.171560 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:25.171631 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:25.172136 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.172674 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.173705 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.174078 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.174185 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:25.177703 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:25.177707 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:25.177709 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:25.177732 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:25.180535 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.181120 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.181123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.181125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.181214 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.181486 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.181492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.181496 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.181707 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:25.181814 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:25.182645 (+ 831us) server_negotiation.cc:299] Negotiation successful
0808 14:31:25.182797 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":134,"threads_started":1}
I20260808 14:31:25.183585 21900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.170522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:43331 (local address 127.0.0.1:47858)
0808 14:31:25.171018 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:25.171033 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:25.171106 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:25.171671 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:25.171674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:25.171843 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:25.171950 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:25.171960 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.172780 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.172786 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:25.173575 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:25.173585 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.174207 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.174210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:25.174293 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:25.174829 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:25.174843 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:25.177566 (+ 2723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:25.180652 (+ 3086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:25.180655 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:25.180657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:25.180968 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:25.181317 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:25.181319 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:25.181321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:25.181400 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:25.181849 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:25.181852 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:25.181947 (+ 95us) client_negotiation.cc:769] Sending connection context
0808 14:31:25.182041 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 14:31:25.182201 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405}
I20260808 14:31:25.185671 21742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.176634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:43331 (local address 127.0.0.1:47862)
0808 14:31:25.176850 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:25.176864 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:25.176943 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:25.177247 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:25.177250 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:25.177432 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:25.177548 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:25.177555 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.178277 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.178281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:25.179146 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:25.179159 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.179754 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.179758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:25.179843 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:25.180302 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:25.180320 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:25.181859 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:25.183976 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:25.183983 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:25.183986 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:25.184301 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:25.184612 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:25.184616 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:25.184618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:25.184715 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:25.185154 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:25.185157 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:25.185253 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 14:31:25.185351 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 14:31:25.185489 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260808 14:31:25.186187 21899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.176666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47862 (local address 127.7.125.124:43331)
0808 14:31:25.177064 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:25.177067 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:25.177076 (+ 9us) server_negotiation.cc:407] Connection header received
0808 14:31:25.177103 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:25.177105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:25.177135 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:25.177225 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:25.177683 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.178161 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.179282 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.179651 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.179749 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:25.181998 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:25.182004 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:25.182011 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:25.182037 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:25.183853 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.184422 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.184425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.184426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.184504 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.184840 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.184844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.184846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.185040 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:25.185868 (+ 828us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:25.185954 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 14:31:25.186059 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334}
I20260808 14:31:25.186503 21818 raft_consensus.cc:348] T 00000000000000000000000000000000 P c71811e534b5473ea654347fb81af3bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991bb3d755524e53b49d34c65bbc3bc9" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33863 } } peers { permanent_uuid: "c71811e534b5473ea654347fb81af3bd" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } } peers { permanent_uuid: "d2939d45413f4547981b10132e02e2b1" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } }
I20260808 14:31:25.186774 21818 raft_consensus.cc:374] T 00000000000000000000000000000000 P c71811e534b5473ea654347fb81af3bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:31:25.187080 21896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1: Bootstrap starting.
I20260808 14:31:25.188426 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:31:25.189388 21739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 991bb3d755524e53b49d34c65bbc3bc9: Bootstrap starting.
I20260808 14:31:25.189894 21896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1: Neither blocks nor log segments found. Creating new log.
I20260808 14:31:25.190805 21896 log.cc:824] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1: Log is configured to *not* fsync() on all Append() calls
I20260808 14:31:25.192233 21739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 991bb3d755524e53b49d34c65bbc3bc9: Neither blocks nor log segments found. Creating new log.
I20260808 14:31:25.193163 21739 log.cc:824] T 00000000000000000000000000000000 P 991bb3d755524e53b49d34c65bbc3bc9: Log is configured to *not* fsync() on all Append() calls
I20260808 14:31:25.193554 21818 raft_consensus.cc:729] T 00000000000000000000000000000000 P c71811e534b5473ea654347fb81af3bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c71811e534b5473ea654347fb81af3bd, State: Initialized, Role: FOLLOWER
I20260808 14:31:25.193755 21896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1: No bootstrap required, opened a new log
I20260808 14:31:25.194199 21818 consensus_queue.cc:260] T 00000000000000000000000000000000 P c71811e534b5473ea654347fb81af3bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991bb3d755524e53b49d34c65bbc3bc9" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33863 } } peers { permanent_uuid: "c71811e534b5473ea654347fb81af3bd" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } } peers { permanent_uuid: "d2939d45413f4547981b10132e02e2b1" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } }
I20260808 14:31:25.195747 21818 sys_catalog.cc:564] T 00000000000000000000000000000000 P c71811e534b5473ea654347fb81af3bd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:31:25.198848 21896 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991bb3d755524e53b49d34c65bbc3bc9" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33863 } } peers { permanent_uuid: "c71811e534b5473ea654347fb81af3bd" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } } peers { permanent_uuid: "d2939d45413f4547981b10132e02e2b1" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } }
I20260808 14:31:25.199275 21896 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:31:25.199389 21896 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2939d45413f4547981b10132e02e2b1, State: Initialized, Role: FOLLOWER
I20260808 14:31:25.200002 21896 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991bb3d755524e53b49d34c65bbc3bc9" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33863 } } peers { permanent_uuid: "c71811e534b5473ea654347fb81af3bd" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } } peers { permanent_uuid: "d2939d45413f4547981b10132e02e2b1" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } }
I20260808 14:31:25.201135 21899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.190877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47874 (local address 127.7.125.124:43331)
0808 14:31:25.191028 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:25.191033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:25.191535 (+ 502us) server_negotiation.cc:407] Connection header received
0808 14:31:25.191713 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:25.191717 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:25.191759 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:25.191808 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:25.192834 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.193553 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.194738 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.195068 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.195145 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:25.197250 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:25.197254 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:25.197263 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:25.197289 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:25.199139 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.199646 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.199650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.199651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.199743 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.200124 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.200127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.200129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.200380 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:25.200483 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:25.200839 (+ 356us) server_negotiation.cc:299] Negotiation successful
0808 14:31:25.200979 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 14:31:25.201680 21741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.191143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45364 (local address 127.7.125.126:33863)
0808 14:31:25.191256 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:25.191263 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:25.191494 (+ 231us) server_negotiation.cc:407] Connection header received
0808 14:31:25.191653 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:25.191657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:25.191707 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:25.191797 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:25.192780 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.193493 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.194704 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.195044 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.195141 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:25.197243 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:25.197247 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:25.197250 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:25.197277 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:25.199567 (+ 2290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.200082 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.200086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.200088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.200179 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.200509 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.200512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.200513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.200718 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:25.200843 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:25.201404 (+ 561us) server_negotiation.cc:299] Negotiation successful
0808 14:31:25.201541 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 14:31:25.202154 21896 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:31:25.199869 21746 master_runner.cc:186] Error getting master registration for 127.7.125.126:33863: OK, instance_id { permanent_uuid: "991bb3d755524e53b49d34c65bbc3bc9" instance_seqno: 1786199484645031 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 14:31:25.204746 21746 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 14:31:25.205494 21924 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991bb3d755524e53b49d34c65bbc3bc9" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33863 } } peers { permanent_uuid: "c71811e534b5473ea654347fb81af3bd" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } } peers { permanent_uuid: "d2939d45413f4547981b10132e02e2b1" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } } }
I20260808 14:31:25.205715 21924 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 14:31:25.206851 21923 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c71811e534b5473ea654347fb81af3bd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:31:25.206933 21923 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:31:25.207142 21912 sys_catalog.cc:455] T 00000000000000000000000000000000 P c71811e534b5473ea654347fb81af3bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991bb3d755524e53b49d34c65bbc3bc9" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33863 } } peers { permanent_uuid: "c71811e534b5473ea654347fb81af3bd" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } } peers { permanent_uuid: "d2939d45413f4547981b10132e02e2b1" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } } }
I20260808 14:31:25.207260 21912 sys_catalog.cc:458] T 00000000000000000000000000000000 P c71811e534b5473ea654347fb81af3bd [sys.catalog]: This master's current role is: FOLLOWER
W20260808 14:31:25.209069 21936 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:31:25.209230 21936 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:31:25.209424 21739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 991bb3d755524e53b49d34c65bbc3bc9: No bootstrap required, opened a new log
I20260808 14:31:25.206836 21825 master_runner.cc:186] Error getting master registration for 127.7.125.126:33863: OK, instance_id { permanent_uuid: "991bb3d755524e53b49d34c65bbc3bc9" instance_seqno: 1786199484645031 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 14:31:25.210546 21825 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 14:31:25.213940 21739 raft_consensus.cc:348] T 00000000000000000000000000000000 P 991bb3d755524e53b49d34c65bbc3bc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991bb3d755524e53b49d34c65bbc3bc9" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33863 } } peers { permanent_uuid: "c71811e534b5473ea654347fb81af3bd" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } } peers { permanent_uuid: "d2939d45413f4547981b10132e02e2b1" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } }
I20260808 14:31:25.214226 21739 raft_consensus.cc:374] T 00000000000000000000000000000000 P 991bb3d755524e53b49d34c65bbc3bc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:31:25.214287 21739 raft_consensus.cc:729] T 00000000000000000000000000000000 P 991bb3d755524e53b49d34c65bbc3bc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 991bb3d755524e53b49d34c65bbc3bc9, State: Initialized, Role: FOLLOWER
I20260808 14:31:25.214838 21739 consensus_queue.cc:260] T 00000000000000000000000000000000 P 991bb3d755524e53b49d34c65bbc3bc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991bb3d755524e53b49d34c65bbc3bc9" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33863 } } peers { permanent_uuid: "c71811e534b5473ea654347fb81af3bd" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } } peers { permanent_uuid: "d2939d45413f4547981b10132e02e2b1" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } }
I20260808 14:31:25.216529 21739 sys_catalog.cc:564] T 00000000000000000000000000000000 P 991bb3d755524e53b49d34c65bbc3bc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:31:25.218660 21823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:25.208276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58884 (local address 127.7.125.125:37525)
0808 14:31:25.208422 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:25.208428 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:25.208448 (+ 20us) server_negotiation.cc:407] Connection header received
0808 14:31:25.208616 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:25.208620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:25.208670 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:25.208740 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:25.209671 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.210419 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.211564 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:25.211905 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:25.212006 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:25.213908 (+ 1902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:25.213911 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:25.213915 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:25.213945 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:25.215946 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.216494 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.216497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.216499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.216598 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:25.217035 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:25.217039 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:25.217040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:25.217280 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:25.217413 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:25.218282 (+ 869us) server_negotiation.cc:299] Negotiation successful
0808 14:31:25.218437 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 14:31:25.219276 21938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 991bb3d755524e53b49d34c65bbc3bc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991bb3d755524e53b49d34c65bbc3bc9" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33863 } } peers { permanent_uuid: "c71811e534b5473ea654347fb81af3bd" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } } peers { permanent_uuid: "d2939d45413f4547981b10132e02e2b1" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } } }
I20260808 14:31:25.220468 21938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 991bb3d755524e53b49d34c65bbc3bc9 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:31:25.228384 21666 master_runner.cc:263] No leader master found from master 991bb3d755524e53b49d34c65bbc3bc9
I20260808 14:31:25.228488 21666 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 14:31:25.235539 21950 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 991bb3d755524e53b49d34c65bbc3bc9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:31:25.235684 21950 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:31:25.247339 21924 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:31:25.247565 21924 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991bb3d755524e53b49d34c65bbc3bc9" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33863 } } peers { permanent_uuid: "c71811e534b5473ea654347fb81af3bd" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } } peers { permanent_uuid: "d2939d45413f4547981b10132e02e2b1" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } }
I20260808 14:31:25.248646 21924 leader_election.cc:288] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 991bb3d755524e53b49d34c65bbc3bc9 (127.7.125.126:33863), c71811e534b5473ea654347fb81af3bd (127.7.125.125:37525)
I20260808 14:31:25.249410 21790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d2939d45413f4547981b10132e02e2b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c71811e534b5473ea654347fb81af3bd" is_pre_election: true
I20260808 14:31:25.249723 21790 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c71811e534b5473ea654347fb81af3bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2939d45413f4547981b10132e02e2b1 in term 0.
I20260808 14:31:25.250344 21842 leader_election.cc:302] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c71811e534b5473ea654347fb81af3bd, d2939d45413f4547981b10132e02e2b1; no voters:
I20260808 14:31:25.251627 21711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d2939d45413f4547981b10132e02e2b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "991bb3d755524e53b49d34c65bbc3bc9" is_pre_election: true
I20260808 14:31:25.251935 21711 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 991bb3d755524e53b49d34c65bbc3bc9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2939d45413f4547981b10132e02e2b1 in term 0.
I20260808 14:31:25.255033 21924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:31:25.255151 21924 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:31:25.255199 21924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:31:25.257041 21924 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991bb3d755524e53b49d34c65bbc3bc9" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33863 } } peers { permanent_uuid: "c71811e534b5473ea654347fb81af3bd" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } } peers { permanent_uuid: "d2939d45413f4547981b10132e02e2b1" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } }
I20260808 14:31:25.257740 21711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d2939d45413f4547981b10132e02e2b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "991bb3d755524e53b49d34c65bbc3bc9"
I20260808 14:31:25.257893 21711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 991bb3d755524e53b49d34c65bbc3bc9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:31:25.259306 21711 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 991bb3d755524e53b49d34c65bbc3bc9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2939d45413f4547981b10132e02e2b1 in term 1.
I20260808 14:31:25.259677 21843 leader_election.cc:302] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991bb3d755524e53b49d34c65bbc3bc9, d2939d45413f4547981b10132e02e2b1; no voters:
I20260808 14:31:25.260425 21790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d2939d45413f4547981b10132e02e2b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c71811e534b5473ea654347fb81af3bd"
I20260808 14:31:25.260581 21790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c71811e534b5473ea654347fb81af3bd [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:31:25.262547 21790 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c71811e534b5473ea654347fb81af3bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2939d45413f4547981b10132e02e2b1 in term 1.
I20260808 14:31:25.265617 21924 leader_election.cc:288] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [CANDIDATE]: Term 1 election: Requested vote from peers 991bb3d755524e53b49d34c65bbc3bc9 (127.7.125.126:33863), c71811e534b5473ea654347fb81af3bd (127.7.125.125:37525)
I20260808 14:31:25.265846 21924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:31:25.265941 21924 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [term 1 LEADER]: Becoming Leader. State: Replica: d2939d45413f4547981b10132e02e2b1, State: Running, Role: LEADER
I20260808 14:31:25.266336 21924 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991bb3d755524e53b49d34c65bbc3bc9" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33863 } } peers { permanent_uuid: "c71811e534b5473ea654347fb81af3bd" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } } peers { permanent_uuid: "d2939d45413f4547981b10132e02e2b1" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } }
I20260808 14:31:25.268360 21924 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2939d45413f4547981b10132e02e2b1. Latest consensus state: current_term: 1 leader_uuid: "d2939d45413f4547981b10132e02e2b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991bb3d755524e53b49d34c65bbc3bc9" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33863 } } peers { permanent_uuid: "c71811e534b5473ea654347fb81af3bd" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } } peers { permanent_uuid: "d2939d45413f4547981b10132e02e2b1" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } } }
I20260808 14:31:25.268482 21924 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [sys.catalog]: This master's current role is: LEADER
I20260808 14:31:25.268869 21954 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:31:25.269852 21954 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:31:25.280429 21790 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c71811e534b5473ea654347fb81af3bd [term 1 FOLLOWER]: Refusing update from remote peer d2939d45413f4547981b10132e02e2b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:31:25.281178 21952 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c71811e534b5473ea654347fb81af3bd" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:31:25.285993 21912 sys_catalog.cc:455] T 00000000000000000000000000000000 P c71811e534b5473ea654347fb81af3bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2939d45413f4547981b10132e02e2b1. Latest consensus state: current_term: 1 leader_uuid: "d2939d45413f4547981b10132e02e2b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991bb3d755524e53b49d34c65bbc3bc9" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33863 } } peers { permanent_uuid: "c71811e534b5473ea654347fb81af3bd" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } } peers { permanent_uuid: "d2939d45413f4547981b10132e02e2b1" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } } }
I20260808 14:31:25.286157 21912 sys_catalog.cc:458] T 00000000000000000000000000000000 P c71811e534b5473ea654347fb81af3bd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:31:25.290064 21711 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 991bb3d755524e53b49d34c65bbc3bc9 [term 1 FOLLOWER]: Refusing update from remote peer d2939d45413f4547981b10132e02e2b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:31:25.290738 21952 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991bb3d755524e53b49d34c65bbc3bc9" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:31:25.295425 21938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 991bb3d755524e53b49d34c65bbc3bc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2939d45413f4547981b10132e02e2b1. Latest consensus state: current_term: 1 leader_uuid: "d2939d45413f4547981b10132e02e2b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991bb3d755524e53b49d34c65bbc3bc9" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33863 } } peers { permanent_uuid: "c71811e534b5473ea654347fb81af3bd" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } } peers { permanent_uuid: "d2939d45413f4547981b10132e02e2b1" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } } }
I20260808 14:31:25.295583 21938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 991bb3d755524e53b49d34c65bbc3bc9 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:31:25.300608 21952 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d2939d45413f4547981b10132e02e2b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991bb3d755524e53b49d34c65bbc3bc9" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33863 } } peers { permanent_uuid: "c71811e534b5473ea654347fb81af3bd" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } } peers { permanent_uuid: "d2939d45413f4547981b10132e02e2b1" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } } }
I20260808 14:31:25.300757 21952 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [sys.catalog]: This master's current role is: LEADER
I20260808 14:31:25.302038 21955 mvcc.cc:205] Tried to move back new op lower bound from 7316273091699699712 to 7316273091654545408. Current Snapshot: MvccSnapshot[applied={T|T < 7316273091699699712}]
I20260808 14:31:25.302382 21938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 991bb3d755524e53b49d34c65bbc3bc9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d2939d45413f4547981b10132e02e2b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991bb3d755524e53b49d34c65bbc3bc9" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33863 } } peers { permanent_uuid: "c71811e534b5473ea654347fb81af3bd" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } } peers { permanent_uuid: "d2939d45413f4547981b10132e02e2b1" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } } }
I20260808 14:31:25.302508 21938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 991bb3d755524e53b49d34c65bbc3bc9 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:31:25.303673 21958 mvcc.cc:205] Tried to move back new op lower bound from 7316273091699699712 to 7316273091654545408. Current Snapshot: MvccSnapshot[applied={T|T < 7316273091699699712}]
I20260808 14:31:25.307875 21924 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d2939d45413f4547981b10132e02e2b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991bb3d755524e53b49d34c65bbc3bc9" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33863 } } peers { permanent_uuid: "c71811e534b5473ea654347fb81af3bd" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } } peers { permanent_uuid: "d2939d45413f4547981b10132e02e2b1" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } } }
I20260808 14:31:25.308022 21924 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1 [sys.catalog]: This master's current role is: LEADER
I20260808 14:31:25.309347 21912 sys_catalog.cc:455] T 00000000000000000000000000000000 P c71811e534b5473ea654347fb81af3bd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d2939d45413f4547981b10132e02e2b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991bb3d755524e53b49d34c65bbc3bc9" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33863 } } peers { permanent_uuid: "c71811e534b5473ea654347fb81af3bd" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 37525 } } peers { permanent_uuid: "d2939d45413f4547981b10132e02e2b1" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 43331 } } }
I20260808 14:31:25.309474 21912 sys_catalog.cc:458] T 00000000000000000000000000000000 P c71811e534b5473ea654347fb81af3bd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:31:25.312067 21956 mvcc.cc:205] Tried to move back new op lower bound from 7316273091699699712 to 7316273091654545408. Current Snapshot: MvccSnapshot[applied={T|T < 7316273091699699712}]
I20260808 14:31:25.312392 21954 catalog_manager.cc:1347] Generated new cluster ID: b59094dce06b4cd29cf7ad49ad2df107
I20260808 14:31:25.312453 21954 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:31:25.329155 21954 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:31:25.330627 21954 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:31:25.347429 21954 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2939d45413f4547981b10132e02e2b1: Generated new TSK 0
I20260808 14:31:25.348376 21954 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:31:26.208284 21923 catalog_manager.cc:1259] Loaded cluster ID: b59094dce06b4cd29cf7ad49ad2df107
I20260808 14:31:26.208370 21923 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c71811e534b5473ea654347fb81af3bd: loading cluster ID for follower catalog manager: success
I20260808 14:31:26.210067 21923 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c71811e534b5473ea654347fb81af3bd: acquiring CA information for follower catalog manager: success
I20260808 14:31:26.211359 21923 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c71811e534b5473ea654347fb81af3bd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 14:31:26.237545 21950 catalog_manager.cc:1259] Loaded cluster ID: b59094dce06b4cd29cf7ad49ad2df107
I20260808 14:31:26.237638 21950 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 991bb3d755524e53b49d34c65bbc3bc9: loading cluster ID for follower catalog manager: success
I20260808 14:31:26.239912 21950 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 991bb3d755524e53b49d34c65bbc3bc9: acquiring CA information for follower catalog manager: success
I20260808 14:31:26.241191 21950 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 991bb3d755524e53b49d34c65bbc3bc9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 14:31:26.409436 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 21666
I20260808 14:31:26.417389 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 21746
I20260808 14:31:26.424820 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 21825
2026-08-08T14:31:26Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3208 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:31:26 dist-test-slave-w206 krb5kdc[21974](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:31:26 dist-test-slave-w206 krb5kdc[21974](info): set up 2 sockets
Aug 08 14:31:26 dist-test-slave-w206 krb5kdc[21974](info): commencing operation
krb5kdc: starting...
W20260808 14:31:27.503553 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:31:27 dist-test-slave-w206 krb5kdc[21974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:31:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:31:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:31:27.639039 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:43915
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:36455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:43915
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:31:27.752218 21988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:27.752563 21988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:27.752663 21988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:27.756752 21988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:31:27.756848 21988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:27.756882 21988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:27.756922 21988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:31:27.756950 21988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:31:27.762403 21988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:36455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:43915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:43915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:27.763630 21988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:27.764757 21988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:27.773766 21994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:27.773938 21988 server_base.cc:1034] running on GCE node
W20260808 14:31:27.773810 21993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:27.774121 21996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:27.774727 21988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:27.776021 21988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:27.777218 21988 hybrid_clock.cc:648] HybridClock initialized: now 1786199487777190 us; error 46 us; skew 500 ppm
Aug 08 14:31:27 dist-test-slave-w206 krb5kdc[21974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199487, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:27.780275 21988 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:31:27.781216 21988 webserver.cc:460] Webserver started at http://127.7.125.126:43815/ using document root <none> and password file <none>
I20260808 14:31:27.781810 21988 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:27.781910 21988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:27.782155 21988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:31:27.784335 21988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "56fdbc7f35d94b9b8fe0b60922601695"
format_stamp: "Formatted at 2026-08-08 14:31:27 on dist-test-slave-w206"
server_key: "667a92076fa75e5731648ac81804edd4"
server_key_iv: "0b8b901391dd59ea1b5c6b8029ef31eb"
server_key_version: "encryptionkey@0"
I20260808 14:31:27.784922 21988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "56fdbc7f35d94b9b8fe0b60922601695"
format_stamp: "Formatted at 2026-08-08 14:31:27 on dist-test-slave-w206"
server_key: "667a92076fa75e5731648ac81804edd4"
server_key_iv: "0b8b901391dd59ea1b5c6b8029ef31eb"
server_key_version: "encryptionkey@0"
I20260808 14:31:27.788842 21988 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:31:27.791286 22004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:27.792479 21988 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 14:31:27.792619 21988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "56fdbc7f35d94b9b8fe0b60922601695"
format_stamp: "Formatted at 2026-08-08 14:31:27 on dist-test-slave-w206"
server_key: "667a92076fa75e5731648ac81804edd4"
server_key_iv: "0b8b901391dd59ea1b5c6b8029ef31eb"
server_key_version: "encryptionkey@0"
I20260808 14:31:27.792737 21988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:27.815478 21988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:27.819083 21988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:27.819314 21988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:27.826740 21988 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:43915
I20260808 14:31:27.826819 22059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:43915 every 8 connection(s)
I20260808 14:31:27.827821 21988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:31:27.830694 22060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:31:27.836562 22060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695: Bootstrap starting.
I20260808 14:31:27.837167 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 21988
I20260808 14:31:27.837337 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:31:27.837656 7669 external_mini_cluster.cc:1428] Setting key 4c50b82d458d747d1b4ea0e2322ec7fe
I20260808 14:31:27.839640 22060 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695: Neither blocks nor log segments found. Creating new log.
I20260808 14:31:27.840581 22060 log.cc:824] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695: Log is configured to *not* fsync() on all Append() calls
I20260808 14:31:27.842911 22060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695: No bootstrap required, opened a new log
Aug 08 14:31:27 dist-test-slave-w206 krb5kdc[21974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:31:27.847278 22060 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56fdbc7f35d94b9b8fe0b60922601695" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43915 } }
I20260808 14:31:27.847570 22060 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:31:27.847657 22060 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56fdbc7f35d94b9b8fe0b60922601695, State: Initialized, Role: FOLLOWER
I20260808 14:31:27.848160 22060 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56fdbc7f35d94b9b8fe0b60922601695" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43915 } }
I20260808 14:31:27.848335 22060 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:31:27.848439 22060 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:31:27.848547 22060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:31:27.849715 22060 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56fdbc7f35d94b9b8fe0b60922601695" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43915 } }
I20260808 14:31:27.850092 22060 leader_election.cc:302] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56fdbc7f35d94b9b8fe0b60922601695; no voters:
I20260808 14:31:27.850481 22060 leader_election.cc:288] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:31:27.850601 22065 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:31:27.850838 22065 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695 [term 1 LEADER]: Becoming Leader. State: Replica: 56fdbc7f35d94b9b8fe0b60922601695, State: Running, Role: LEADER
I20260808 14:31:27.851186 22065 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56fdbc7f35d94b9b8fe0b60922601695" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43915 } }
I20260808 14:31:27.851953 22060 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:31:27.852394 22065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56fdbc7f35d94b9b8fe0b60922601695. Latest consensus state: current_term: 1 leader_uuid: "56fdbc7f35d94b9b8fe0b60922601695" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56fdbc7f35d94b9b8fe0b60922601695" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43915 } } }
I20260808 14:31:27.852535 22065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695 [sys.catalog]: This master's current role is: LEADER
I20260808 14:31:27.853037 22066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56fdbc7f35d94b9b8fe0b60922601695" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56fdbc7f35d94b9b8fe0b60922601695" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43915 } } }
I20260808 14:31:27.853144 22066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695 [sys.catalog]: This master's current role is: LEADER
I20260808 14:31:27.853835 22063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:27.838954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55930 (local address 127.7.125.126:43915)
0808 14:31:27.839472 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:27.839502 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:27.839547 (+ 45us) server_negotiation.cc:407] Connection header received
0808 14:31:27.840494 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:27.840518 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:27.840945 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:27.841270 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:27.842082 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:27.843587 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:27.844487 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:27.845208 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:27.845373 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:27.848093 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:27.848098 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:27.848113 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:27.848141 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:27.850060 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:27.850559 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:27.850567 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:27.850574 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:27.850818 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:27.851202 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:27.851204 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:27.851205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:27.851670 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:27.851898 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:27.852259 (+ 361us) server_negotiation.cc:299] Negotiation successful
0808 14:31:27.852701 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":128,"threads_started":1}
I20260808 14:31:27.857383 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 14:31:27.857417 22081 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:31:27.857486 22081 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:31:27.857548 22069 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:31:27.858528 22069 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:31:27.864709 22069 catalog_manager.cc:1347] Generated new cluster ID: 0f478914dbc64d7aa3039f41b3dc0923
I20260808 14:31:27.864795 22069 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:31:27.887872 22069 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:31:27.889246 22069 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:31:27.895362 22069 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56fdbc7f35d94b9b8fe0b60922601695: Generated new TSK 0
I20260808 14:31:27.896130 22069 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:31:27.914273 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 21988
2026-08-08T14:31:27Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1488 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:31:27 dist-test-slave-w206 krb5kdc[22088](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:31:27 dist-test-slave-w206 krb5kdc[22088](info): set up 2 sockets
Aug 08 14:31:27 dist-test-slave-w206 krb5kdc[22088](info): commencing operation
krb5kdc: starting...
W20260808 14:31:28.988451 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:31:29 dist-test-slave-w206 krb5kdc[22088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:31:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:31:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:31:29.122506 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:43375
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:43435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:43375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:31:29.237887 22104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:29.238225 22104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:29.238312 22104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:29.242491 22104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:31:29.242599 22104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:29.242635 22104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:29.242678 22104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:31:29.242712 22104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:31:29.248292 22104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:43435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:43375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:43375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:29.249420 22104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:29.250479 22104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:29.259024 22110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:29.259032 22109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:29.259004 22112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:29.259503 22104 server_base.cc:1034] running on GCE node
I20260808 14:31:29.260082 22104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:29.261229 22104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:29.262390 22104 hybrid_clock.cc:648] HybridClock initialized: now 1786199489262369 us; error 32 us; skew 500 ppm
Aug 08 14:31:29 dist-test-slave-w206 krb5kdc[22088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199489, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:29.271163 22104 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:31:29.272264 22104 webserver.cc:460] Webserver started at http://127.7.125.126:34677/ using document root <none> and password file <none>
I20260808 14:31:29.272887 22104 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:29.272955 22104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:29.273144 22104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:31:29.275627 22104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "17fe17e03d7a498897f8d082462f118a"
format_stamp: "Formatted at 2026-08-08 14:31:29 on dist-test-slave-w206"
server_key: "9b48449ff31d8f7a53ba7b76e3fc0a92"
server_key_iv: "55da9ad9f0a4fa8d8c532ae23fa2a9bf"
server_key_version: "encryptionkey@0"
I20260808 14:31:29.276204 22104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "17fe17e03d7a498897f8d082462f118a"
format_stamp: "Formatted at 2026-08-08 14:31:29 on dist-test-slave-w206"
server_key: "9b48449ff31d8f7a53ba7b76e3fc0a92"
server_key_iv: "55da9ad9f0a4fa8d8c532ae23fa2a9bf"
server_key_version: "encryptionkey@0"
I20260808 14:31:29.280550 22104 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:31:29.283321 22119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:29.284732 22104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 14:31:29.284953 22104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "17fe17e03d7a498897f8d082462f118a"
format_stamp: "Formatted at 2026-08-08 14:31:29 on dist-test-slave-w206"
server_key: "9b48449ff31d8f7a53ba7b76e3fc0a92"
server_key_iv: "55da9ad9f0a4fa8d8c532ae23fa2a9bf"
server_key_version: "encryptionkey@0"
I20260808 14:31:29.285072 22104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:29.333730 22104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:29.337862 22104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:29.338049 22104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:29.346139 22104 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:43375
I20260808 14:31:29.346429 22171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:43375 every 8 connection(s)
I20260808 14:31:29.347293 22104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:31:29.350147 22172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:31:29.351213 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 22104
I20260808 14:31:29.351387 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:31:29.351729 7669 external_mini_cluster.cc:1428] Setting key b1626eb5d937a5507990515cc9d620b8
I20260808 14:31:29.356740 22172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a: Bootstrap starting.
I20260808 14:31:29.359545 22172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a: Neither blocks nor log segments found. Creating new log.
I20260808 14:31:29.360522 22172 log.cc:824] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:31:29 dist-test-slave-w206 krb5kdc[22088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:31:29.362718 22172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a: No bootstrap required, opened a new log
I20260808 14:31:29.366429 22172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fe17e03d7a498897f8d082462f118a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43375 } }
I20260808 14:31:29.366655 22172 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:31:29.366724 22172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fe17e03d7a498897f8d082462f118a, State: Initialized, Role: FOLLOWER
I20260808 14:31:29.367136 22176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:29.353255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56446 (local address 127.7.125.126:43375)
0808 14:31:29.353729 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:29.353747 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:29.353786 (+ 39us) server_negotiation.cc:407] Connection header received
0808 14:31:29.354475 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:29.354493 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:29.354853 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:29.355241 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:29.356265 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:29.357420 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:29.358657 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:29.359271 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:29.359405 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:29.362296 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:29.362301 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:29.362314 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:29.362341 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:29.364129 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:29.364590 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:29.364595 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:29.364599 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:29.364708 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:29.365012 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:29.365016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:29.365018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:29.365420 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:29.365645 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:29.365926 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 14:31:29.366216 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":153,"threads_started":1}
I20260808 14:31:29.367218 22172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fe17e03d7a498897f8d082462f118a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43375 } }
I20260808 14:31:29.367446 22172 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:31:29.367529 22172 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:31:29.367657 22172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:31:29.368705 22172 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fe17e03d7a498897f8d082462f118a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43375 } }
I20260808 14:31:29.369055 22172 leader_election.cc:302] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17fe17e03d7a498897f8d082462f118a; no voters:
I20260808 14:31:29.369421 22172 leader_election.cc:288] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:31:29.369554 22180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:31:29.369812 22180 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 1 LEADER]: Becoming Leader. State: Replica: 17fe17e03d7a498897f8d082462f118a, State: Running, Role: LEADER
I20260808 14:31:29.370149 22180 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fe17e03d7a498897f8d082462f118a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43375 } }
I20260808 14:31:29.370666 22172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:31:29.371745 22181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17fe17e03d7a498897f8d082462f118a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fe17e03d7a498897f8d082462f118a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43375 } } }
I20260808 14:31:29.371896 22181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [sys.catalog]: This master's current role is: LEADER
I20260808 14:31:29.372229 22182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17fe17e03d7a498897f8d082462f118a. Latest consensus state: current_term: 1 leader_uuid: "17fe17e03d7a498897f8d082462f118a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fe17e03d7a498897f8d082462f118a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43375 } } }
I20260808 14:31:29.372324 22190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:31:29.372352 22182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [sys.catalog]: This master's current role is: LEADER
I20260808 14:31:29.375447 22190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:31:29.381134 22190 catalog_manager.cc:1347] Generated new cluster ID: bb89ed74fe1d4d71973af55096723866
I20260808 14:31:29.381215 22190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:31:29.389004 22190 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:31:29.390079 22190 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:31:29.398018 22190 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a: Generated new TSK 0
I20260808 14:31:29.398696 22190 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.125.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
I20260808 14:31:29.436360 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:43375
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:43435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:31:29.559633 22203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:29.559934 22203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:29.559979 22203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:29.564129 22203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:29.564194 22203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:29.564254 22203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:31:29.569556 22203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:43435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:43375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:29.570596 22203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:29.571787 22203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:29.583103 22211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:29.583130 22209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:29.583351 22203 server_base.cc:1034] running on GCE node
W20260808 14:31:29.583500 22208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:29.584069 22203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:29.584775 22203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:29.585968 22203 hybrid_clock.cc:648] HybridClock initialized: now 1786199489585953 us; error 29 us; skew 500 ppm
Aug 08 14:31:29 dist-test-slave-w206 krb5kdc[22088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199489, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:29.588876 22203 init.cc:377] Logged in from keytab as kudu/127.7.125.65@KRBTEST.COM (short username kudu)
I20260808 14:31:29.589895 22203 webserver.cc:460] Webserver started at http://127.7.125.65:32795/ using document root <none> and password file <none>
I20260808 14:31:29.590565 22203 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:29.590660 22203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:29.590898 22203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:31:29.593036 22203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "2bf980017525452096a42bbd2ef96c08"
format_stamp: "Formatted at 2026-08-08 14:31:29 on dist-test-slave-w206"
server_key: "bdc5e36189de974eaa6bc4473d836da4"
server_key_iv: "aa9bfa2c378f6808fe240b0b56879196"
server_key_version: "encryptionkey@0"
I20260808 14:31:29.593626 22203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "2bf980017525452096a42bbd2ef96c08"
format_stamp: "Formatted at 2026-08-08 14:31:29 on dist-test-slave-w206"
server_key: "bdc5e36189de974eaa6bc4473d836da4"
server_key_iv: "aa9bfa2c378f6808fe240b0b56879196"
server_key_version: "encryptionkey@0"
I20260808 14:31:29.597491 22203 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260808 14:31:29.600015 22220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:29.601285 22203 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:31:29.601439 22203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "2bf980017525452096a42bbd2ef96c08"
format_stamp: "Formatted at 2026-08-08 14:31:29 on dist-test-slave-w206"
server_key: "bdc5e36189de974eaa6bc4473d836da4"
server_key_iv: "aa9bfa2c378f6808fe240b0b56879196"
server_key_version: "encryptionkey@0"
I20260808 14:31:29.601557 22203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:29.624966 22203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:29.628119 22203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:29.628341 22203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:29.628984 22203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:31:29.629870 22203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:31:29.629948 22203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:29.630025 22203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:31:29.630070 22203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:29.639279 22203 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:43577
I20260808 14:31:29.639302 22342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:43577 every 8 connection(s)
I20260808 14:31:29.640237 22203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:31:29.645153 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 22203
I20260808 14:31:29.645272 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:31:29.645584 7669 external_mini_cluster.cc:1428] Setting key 97efc94ba3f4bd648041ee6d17a9478e
Aug 08 14:31:29 dist-test-slave-w206 krb5kdc[22088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199489, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:31:29.655206 22176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:29.642084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:38923 (local address 127.7.125.126:43375)
0808 14:31:29.642221 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:29.642225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:29.643077 (+ 852us) server_negotiation.cc:407] Connection header received
0808 14:31:29.644312 (+ 1235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:29.644315 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:29.644360 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:29.644438 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:29.645722 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:29.646421 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:29.647786 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:29.648191 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:29.648314 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:29.651284 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:29.651287 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:29.651289 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:29.651313 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:29.652816 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:29.653500 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:29.653503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:29.653504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:29.653577 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:29.653992 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:29.653994 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:29.653996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:29.654165 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:29.654236 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:29.654913 (+ 677us) server_negotiation.cc:299] Negotiation successful
0808 14:31:29.655081 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:31:29.656456 22345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:29.642452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.65:38923)
0808 14:31:29.642884 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:29.642929 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:29.644125 (+ 1196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:29.644597 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:29.644604 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:29.644957 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:29.645543 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:29.645559 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:29.646555 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:29.646559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:29.647657 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:29.647667 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:29.648350 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:29.648352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:29.648501 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:29.649168 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:29.649184 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:29.651103 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:29.652944 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:29.652954 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:29.652973 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:29.653377 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:29.653671 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:29.653675 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:29.653679 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:29.653891 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:29.654273 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:29.654281 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:29.654641 (+ 360us) client_negotiation.cc:769] Sending connection context
0808 14:31:29.654867 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 14:31:29.655276 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for kudu/127.7.125.66@KRBTEST.COM; defaulting to no policy
I20260808 14:31:29.657838 22343 heartbeater.cc:347] Connected to a master server at 127.7.125.126:43375
I20260808 14:31:29.658128 22343 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:29.658650 22343 heartbeater.cc:511] Master 127.7.125.126:43375 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.66@KRBTEST.COM" created.
I20260808 14:31:29.660394 22136 ts_manager.cc:194] Registered new tserver with Master: 2bf980017525452096a42bbd2ef96c08 (127.7.125.65:43577)
I20260808 14:31:29.661511 22136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:38923
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
I20260808 14:31:29.677672 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:43375
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:43435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:31:29.791254 22348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:29.791507 22348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:29.791584 22348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:29.795722 22348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:29.795778 22348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:29.795864 22348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:31:29.801167 22348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:43435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:43375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:29.802273 22348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:29.803344 22348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:29.813097 22356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:29.813163 22354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:29.813097 22353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:29.813372 22348 server_base.cc:1034] running on GCE node
I20260808 14:31:29.813953 22348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:29.814667 22348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:29.815860 22348 hybrid_clock.cc:648] HybridClock initialized: now 1786199489815821 us; error 56 us; skew 500 ppm
Aug 08 14:31:29 dist-test-slave-w206 krb5kdc[22088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199489, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:29.818589 22348 init.cc:377] Logged in from keytab as kudu/127.7.125.66@KRBTEST.COM (short username kudu)
I20260808 14:31:29.819579 22348 webserver.cc:460] Webserver started at http://127.7.125.66:32937/ using document root <none> and password file <none>
I20260808 14:31:29.820165 22348 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:29.820264 22348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:29.820510 22348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:31:29.822592 22348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "9c2714e1b2cf4ada877e1c1a992c61a0"
format_stamp: "Formatted at 2026-08-08 14:31:29 on dist-test-slave-w206"
server_key: "941c2c7c1969bfbc408b56d3949c96da"
server_key_iv: "e2d3a92a2a2d5621794d8117060f591c"
server_key_version: "encryptionkey@0"
I20260808 14:31:29.823237 22348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "9c2714e1b2cf4ada877e1c1a992c61a0"
format_stamp: "Formatted at 2026-08-08 14:31:29 on dist-test-slave-w206"
server_key: "941c2c7c1969bfbc408b56d3949c96da"
server_key_iv: "e2d3a92a2a2d5621794d8117060f591c"
server_key_version: "encryptionkey@0"
I20260808 14:31:29.826920 22348 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:31:29.829363 22363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:29.830575 22348 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 14:31:29.830718 22348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "9c2714e1b2cf4ada877e1c1a992c61a0"
format_stamp: "Formatted at 2026-08-08 14:31:29 on dist-test-slave-w206"
server_key: "941c2c7c1969bfbc408b56d3949c96da"
server_key_iv: "e2d3a92a2a2d5621794d8117060f591c"
server_key_version: "encryptionkey@0"
I20260808 14:31:29.830837 22348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:29.840272 22348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:29.843174 22348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:29.843382 22348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:29.843981 22348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:31:29.844820 22348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:31:29.844893 22348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:29.844956 22348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:31:29.845006 22348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:29.854291 22348 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:42295
I20260808 14:31:29.854321 22485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:42295 every 8 connection(s)
I20260808 14:31:29.855275 22348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:31:29.864900 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 22348
I20260808 14:31:29.865056 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
Aug 08 14:31:29 dist-test-slave-w206 krb5kdc[22088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199489, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:31:29.865433 7669 external_mini_cluster.cc:1428] Setting key be360656334395966aa17cf9beb6bcf0
I20260808 14:31:29.869923 22176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:29.857120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:45463 (local address 127.7.125.126:43375)
0808 14:31:29.857315 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:29.857319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:29.858114 (+ 795us) server_negotiation.cc:407] Connection header received
0808 14:31:29.859562 (+ 1448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:29.859565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:29.859628 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:29.859739 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:29.860896 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:29.861426 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:29.862619 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:29.862871 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:29.862957 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:29.865914 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:29.865918 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:29.865921 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:29.865953 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:29.867557 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:29.868229 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:29.868232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:29.868233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:29.868310 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:29.868707 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:29.868709 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:29.868710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:29.868883 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:29.868959 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:29.869626 (+ 667us) server_negotiation.cc:299] Negotiation successful
0808 14:31:29.869759 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 14:31:29.870920 22488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:29.857518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.66:45463)
0808 14:31:29.857958 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:29.858001 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:29.859328 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:29.859908 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:29.859926 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:29.860292 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:29.860750 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:29.860764 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:29.861590 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:29.861593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:29.862486 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:29.862495 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:29.863068 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:29.863072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:29.863217 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:29.863876 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:29.863893 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:29.865708 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:29.867682 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:29.867691 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:29.867709 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:29.868121 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:29.868397 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:29.868403 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:29.868408 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:29.868609 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:29.868991 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:29.869000 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:29.869359 (+ 359us) client_negotiation.cc:769] Sending connection context
0808 14:31:29.869575 (+ 216us) client_negotiation.cc:240] Negotiation successful
0808 14:31:29.869921 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":108,"threads_started":1}
I20260808 14:31:29.872180 22486 heartbeater.cc:347] Connected to a master server at 127.7.125.126:43375
I20260808 14:31:29.872455 22486 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:29.873135 22486 heartbeater.cc:511] Master 127.7.125.126:43375 requested a full tablet report, sending...
I20260808 14:31:29.874457 22136 ts_manager.cc:194] Registered new tserver with Master: 9c2714e1b2cf4ada877e1c1a992c61a0 (127.7.125.66:42295)
I20260808 14:31:29.875372 22136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:45463
WARNING: no policy specified for kudu/127.7.125.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
I20260808 14:31:29.898254 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:43375
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:43435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:31:30.012382 22491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:30.012655 22491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:30.012702 22491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:30.016848 22491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:30.016903 22491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:30.016963 22491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:31:30.022284 22491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:43435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:43375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:30.023412 22491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:30.024405 22491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:31:30.034438 22491 server_base.cc:1034] running on GCE node
W20260808 14:31:30.034363 22497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:30.034358 22498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:30.034406 22500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:30.035207 22491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:30.035939 22491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:30.037104 22491 hybrid_clock.cc:648] HybridClock initialized: now 1786199490037086 us; error 29 us; skew 500 ppm
Aug 08 14:31:30 dist-test-slave-w206 krb5kdc[22088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199490, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:30.040175 22491 init.cc:377] Logged in from keytab as kudu/127.7.125.67@KRBTEST.COM (short username kudu)
I20260808 14:31:30.041100 22491 webserver.cc:460] Webserver started at http://127.7.125.67:39557/ using document root <none> and password file <none>
I20260808 14:31:30.041684 22491 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:30.041746 22491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:30.041985 22491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:31:30.044138 22491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "135b2386669b4430a003c3cf6dbb7728"
format_stamp: "Formatted at 2026-08-08 14:31:30 on dist-test-slave-w206"
server_key: "b21110fd0531d3fa3825ca7841d782dd"
server_key_iv: "1c8f7a1c7540a29ca6f1d58fade22d8e"
server_key_version: "encryptionkey@0"
I20260808 14:31:30.044739 22491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "135b2386669b4430a003c3cf6dbb7728"
format_stamp: "Formatted at 2026-08-08 14:31:30 on dist-test-slave-w206"
server_key: "b21110fd0531d3fa3825ca7841d782dd"
server_key_iv: "1c8f7a1c7540a29ca6f1d58fade22d8e"
server_key_version: "encryptionkey@0"
I20260808 14:31:30.048578 22491 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 14:31:30.050962 22507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:30.052260 22491 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 14:31:30.052398 22491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "135b2386669b4430a003c3cf6dbb7728"
format_stamp: "Formatted at 2026-08-08 14:31:30 on dist-test-slave-w206"
server_key: "b21110fd0531d3fa3825ca7841d782dd"
server_key_iv: "1c8f7a1c7540a29ca6f1d58fade22d8e"
server_key_version: "encryptionkey@0"
I20260808 14:31:30.052505 22491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:30.067724 22491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:30.070786 22491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:30.071060 22491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:30.071756 22491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:31:30.072676 22491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:31:30.072748 22491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:30.072819 22491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:31:30.072867 22491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:30.081964 22491 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:44891
I20260808 14:31:30.081986 22624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:44891 every 8 connection(s)
I20260808 14:31:30.082868 22491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:31:30.085783 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 22491
I20260808 14:31:30.085952 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:31:30.086279 7669 external_mini_cluster.cc:1428] Setting key 983b3ad72f1bf9d0120fe0526bfda8f7
Aug 08 14:31:30 dist-test-slave-w206 krb5kdc[22088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199490, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:31:30.097421 22176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:30.084731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:43471 (local address 127.7.125.126:43375)
0808 14:31:30.084911 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:30.084916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:30.085898 (+ 982us) server_negotiation.cc:407] Connection header received
0808 14:31:30.087633 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:30.087639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:30.087701 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:30.087779 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:30.089120 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.089722 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.090943 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.091224 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.091337 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:30.093887 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:30.093890 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:30.093892 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:30.093925 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:30.095312 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:30.095841 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:30.095844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:30.095846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:30.095940 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:30.096298 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:30.096303 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:30.096305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:30.096499 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:30.096612 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:30.097119 (+ 507us) server_negotiation.cc:299] Negotiation successful
0808 14:31:30.097250 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 14:31:30.098304 22628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:30.085155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.67:43471)
0808 14:31:30.085690 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:30.085754 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:30.087386 (+ 1632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:30.087987 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:30.087994 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:30.088395 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:30.088990 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:30.089004 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.089832 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.089835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:30.090795 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:30.090805 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.091381 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.091385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:30.091567 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:30.092205 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:30.092222 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:30.093703 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:30.095436 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:30.095442 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:30.095459 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:30.095729 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:30.096045 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:30.096049 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:30.096051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:30.096199 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:30.096610 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:30.096618 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:30.096876 (+ 258us) client_negotiation.cc:769] Sending connection context
0808 14:31:30.097078 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 14:31:30.097440 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":116,"threads_started":1}
I20260808 14:31:30.099743 22626 heartbeater.cc:347] Connected to a master server at 127.7.125.126:43375
I20260808 14:31:30.100036 22626 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:30.100616 22626 heartbeater.cc:511] Master 127.7.125.126:43375 requested a full tablet report, sending...
I20260808 14:31:30.101636 22136 ts_manager.cc:194] Registered new tserver with Master: 135b2386669b4430a003c3cf6dbb7728 (127.7.125.67:44891)
I20260808 14:31:30.102223 22136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:43471
I20260808 14:31:30.112810 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:31:30.122563 22176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:30.114927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56448 (local address 127.7.125.126:43375)
0808 14:31:30.115082 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:30.115086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:30.115352 (+ 266us) server_negotiation.cc:407] Connection header received
0808 14:31:30.115557 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:30.115560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:30.115615 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:30.115708 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:30.116450 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.116981 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.117826 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.118058 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.118168 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:30.119569 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:30.119572 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:30.119575 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:30.119610 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:30.120837 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:30.121275 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:30.121280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:30.121282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:30.121368 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:30.121650 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:30.121654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:30.121655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:30.121887 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:30.122047 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:30.122278 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 14:31:30.122393 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 14:31:30.125437 22136 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56448:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:31:30.128624 22136 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:31:30.143945 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:30.138288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49172 (local address 127.7.125.66:42295)
0808 14:31:30.139260 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:30.139272 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:30.139294 (+ 22us) server_negotiation.cc:407] Connection header received
0808 14:31:30.139381 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:30.139389 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:30.139547 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:30.139698 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:31:30.139956 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.140854 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.142566 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.143208 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.143329 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:30.143504 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:30.143679 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 14:31:30.143776 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":867,"thread_start_us":137,"threads_started":1}
I20260808 14:31:30.143935 22639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:30.138127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:42295 (local address 127.0.0.1:49172)
0808 14:31:30.138477 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:30.138490 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:30.138571 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:30.139676 (+ 1105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:30.139679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:30.139696 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:31:30.139804 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:30.139811 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.141017 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.141020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:30.142411 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:30.142423 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.143331 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.143334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:30.143449 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:30.143505 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 14:31:30.143646 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 14:31:30.143776 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":69,"thread_start_us":100,"threads_started":1}
I20260808 14:31:30.144192 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:30.137494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38386 (local address 127.7.125.67:44891)
0808 14:31:30.137893 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:30.137909 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:30.138546 (+ 637us) server_negotiation.cc:407] Connection header received
0808 14:31:30.138634 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:30.138640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:30.138885 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:30.139086 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:31:30.139418 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.140660 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.142538 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.143367 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.143469 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:30.143604 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:30.143918 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 14:31:30.144028 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":169,"threads_started":1}
I20260808 14:31:30.144260 22641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:30.138455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:43577 (local address 127.0.0.1:42686)
0808 14:31:30.138864 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:30.138875 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:30.138953 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:30.139871 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:30.139877 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:30.139897 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:31:30.139999 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:30.140010 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.141317 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.141319 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:30.142253 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:30.142263 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.143616 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.143622 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:30.143713 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:30.143753 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 14:31:30.143828 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 14:31:30.143925 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":55,"thread_start_us":86,"threads_started":1}
I20260808 14:31:30.144681 22642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:30.138858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42686 (local address 127.7.125.65:43577)
0808 14:31:30.139408 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:30.139423 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:30.139444 (+ 21us) server_negotiation.cc:407] Connection header received
0808 14:31:30.139516 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:30.139521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:30.139695 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:30.139835 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:31:30.140126 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.141161 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.142382 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.143273 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.143377 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:30.143550 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:30.144012 (+ 462us) server_negotiation.cc:299] Negotiation successful
0808 14:31:30.144542 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":103,"threads_started":1}
I20260808 14:31:30.144732 22637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:30.137341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:44891 (local address 127.0.0.1:38386)
0808 14:31:30.138067 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:30.138224 (+ 157us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:30.138362 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:30.139100 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:30.139108 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:30.139132 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:31:30.139279 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:30.139288 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.140805 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.140809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:30.142420 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:30.142427 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.143651 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.143654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:30.143727 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:30.143792 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 14:31:30.144522 (+ 730us) client_negotiation.cc:240] Negotiation successful
0808 14:31:30.144593 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"mutex_wait_us":170,"thread_start_us":112,"threads_started":1}
I20260808 14:31:30.147575 22559 tablet_service.cc:1467] Processing CreateTablet for tablet 76512cfb0447438084827a10baa0f662 (DEFAULT_TABLE table=test-table [id=0c4f887c4749486aace5ec5ddf10f837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:31:30.147560 22272 tablet_service.cc:1467] Processing CreateTablet for tablet 76512cfb0447438084827a10baa0f662 (DEFAULT_TABLE table=test-table [id=0c4f887c4749486aace5ec5ddf10f837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:31:30.147573 22418 tablet_service.cc:1467] Processing CreateTablet for tablet 76512cfb0447438084827a10baa0f662 (DEFAULT_TABLE table=test-table [id=0c4f887c4749486aace5ec5ddf10f837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:31:30.148685 22418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76512cfb0447438084827a10baa0f662. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:31:30.148694 22559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76512cfb0447438084827a10baa0f662. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:31:30.148689 22272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76512cfb0447438084827a10baa0f662. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:31:30.155207 22643 tablet_bootstrap.cc:492] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08: Bootstrap starting.
I20260808 14:31:30.156421 22645 tablet_bootstrap.cc:492] T 76512cfb0447438084827a10baa0f662 P 135b2386669b4430a003c3cf6dbb7728: Bootstrap starting.
I20260808 14:31:30.157572 22643 tablet_bootstrap.cc:654] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08: Neither blocks nor log segments found. Creating new log.
I20260808 14:31:30.158332 22643 log.cc:824] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08: Log is configured to *not* fsync() on all Append() calls
I20260808 14:31:30.158488 22645 tablet_bootstrap.cc:654] T 76512cfb0447438084827a10baa0f662 P 135b2386669b4430a003c3cf6dbb7728: Neither blocks nor log segments found. Creating new log.
I20260808 14:31:30.159387 22644 tablet_bootstrap.cc:492] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0: Bootstrap starting.
I20260808 14:31:30.159775 22645 log.cc:824] T 76512cfb0447438084827a10baa0f662 P 135b2386669b4430a003c3cf6dbb7728: Log is configured to *not* fsync() on all Append() calls
I20260808 14:31:30.160663 22643 tablet_bootstrap.cc:492] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08: No bootstrap required, opened a new log
I20260808 14:31:30.160866 22643 ts_tablet_manager.cc:1397] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:31:30.161579 22645 tablet_bootstrap.cc:492] T 76512cfb0447438084827a10baa0f662 P 135b2386669b4430a003c3cf6dbb7728: No bootstrap required, opened a new log
I20260808 14:31:30.161816 22645 ts_tablet_manager.cc:1397] T 76512cfb0447438084827a10baa0f662 P 135b2386669b4430a003c3cf6dbb7728: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:31:30.162098 22644 tablet_bootstrap.cc:654] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0: Neither blocks nor log segments found. Creating new log.
I20260808 14:31:30.162966 22644 log.cc:824] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0: Log is configured to *not* fsync() on all Append() calls
I20260808 14:31:30.165220 22644 tablet_bootstrap.cc:492] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0: No bootstrap required, opened a new log
I20260808 14:31:30.165056 22645 raft_consensus.cc:348] T 76512cfb0447438084827a10baa0f662 P 135b2386669b4430a003c3cf6dbb7728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:30.165324 22645 raft_consensus.cc:374] T 76512cfb0447438084827a10baa0f662 P 135b2386669b4430a003c3cf6dbb7728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:31:30.165401 22645 raft_consensus.cc:729] T 76512cfb0447438084827a10baa0f662 P 135b2386669b4430a003c3cf6dbb7728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 135b2386669b4430a003c3cf6dbb7728, State: Initialized, Role: FOLLOWER
I20260808 14:31:30.165228 22643 raft_consensus.cc:348] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:30.165444 22644 ts_tablet_manager.cc:1397] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:31:30.165540 22643 raft_consensus.cc:374] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:31:30.165602 22643 raft_consensus.cc:729] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bf980017525452096a42bbd2ef96c08, State: Initialized, Role: FOLLOWER
I20260808 14:31:30.165910 22645 consensus_queue.cc:260] T 76512cfb0447438084827a10baa0f662 P 135b2386669b4430a003c3cf6dbb7728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:30.166095 22643 consensus_queue.cc:260] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:30.166960 22645 ts_tablet_manager.cc:1428] T 76512cfb0447438084827a10baa0f662 P 135b2386669b4430a003c3cf6dbb7728: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 14:31:30.167079 22643 ts_tablet_manager.cc:1428] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260808 14:31:30.167407 22626 heartbeater.cc:503] Master 127.7.125.126:43375 was elected leader, sending a full tablet report...
I20260808 14:31:30.167419 22343 heartbeater.cc:503] Master 127.7.125.126:43375 was elected leader, sending a full tablet report...
I20260808 14:31:30.169188 22644 raft_consensus.cc:348] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:30.169461 22644 raft_consensus.cc:374] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:31:30.169543 22644 raft_consensus.cc:729] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2714e1b2cf4ada877e1c1a992c61a0, State: Initialized, Role: FOLLOWER
I20260808 14:31:30.169982 22644 consensus_queue.cc:260] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:30.170689 22486 heartbeater.cc:503] Master 127.7.125.126:43375 was elected leader, sending a full tablet report...
I20260808 14:31:30.171047 22644 ts_tablet_manager.cc:1428] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 14:31:30.245752 22653 raft_consensus.cc:482] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:31:30.245985 22653 raft_consensus.cc:504] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:30.247171 22653 leader_election.cc:288] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 135b2386669b4430a003c3cf6dbb7728 (127.7.125.67:44891), 2bf980017525452096a42bbd2ef96c08 (127.7.125.65:43577)
I20260808 14:31:30.251437 22488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:30.247441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:44891 (local address 127.7.125.66:40335)
0808 14:31:30.247568 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:30.247585 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:30.247687 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:30.247993 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:30.247996 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:30.248017 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:31:30.248146 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:30.248154 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.249142 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.249144 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:30.250283 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:30.250293 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.251075 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.251077 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:30.251155 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:30.251175 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 14:31:30.251236 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 14:31:30.251292 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 14:31:30.251489 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:30.247562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:40335 (local address 127.7.125.67:44891)
0808 14:31:30.247693 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:30.247698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:30.247717 (+ 19us) server_negotiation.cc:407] Connection header received
0808 14:31:30.247779 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:30.247784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:30.247836 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:30.247965 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:31:30.248267 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.249036 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.250414 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.250929 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.251067 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:30.251121 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:30.251279 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 14:31:30.251351 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 14:31:30.251974 22654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:30.247728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:43577 (local address 127.7.125.66:47541)
0808 14:31:30.248222 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:30.248237 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:30.248327 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:30.248554 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:30.248556 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:30.248569 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:31:30.248665 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:30.248669 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.249509 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.249512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:30.250572 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:30.250579 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.251481 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.251488 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:30.251585 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:30.251623 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 14:31:30.251755 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 14:31:30.251840 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":1,"thread_start_us":182,"threads_started":1}
I20260808 14:31:30.251981 22642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:30.247933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:47541 (local address 127.7.125.65:43577)
0808 14:31:30.248067 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:30.248072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:30.248324 (+ 252us) server_negotiation.cc:407] Connection header received
0808 14:31:30.248398 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:30.248401 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:30.248451 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:30.248538 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:31:30.248770 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.249389 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.250687 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.251327 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.251441 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:30.251484 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:30.251777 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 14:31:30.251869 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:31:30.252180 22579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76512cfb0447438084827a10baa0f662" candidate_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "135b2386669b4430a003c3cf6dbb7728" is_pre_election: true
I20260808 14:31:30.252530 22579 raft_consensus.cc:2454] T 76512cfb0447438084827a10baa0f662 P 135b2386669b4430a003c3cf6dbb7728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2714e1b2cf4ada877e1c1a992c61a0 in term 0.
I20260808 14:31:30.252599 22293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76512cfb0447438084827a10baa0f662" candidate_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bf980017525452096a42bbd2ef96c08" is_pre_election: true
I20260808 14:31:30.252874 22293 raft_consensus.cc:2454] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2714e1b2cf4ada877e1c1a992c61a0 in term 0.
I20260808 14:31:30.253054 22365 leader_election.cc:302] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 135b2386669b4430a003c3cf6dbb7728, 9c2714e1b2cf4ada877e1c1a992c61a0; no voters:
I20260808 14:31:30.253374 22653 raft_consensus.cc:2781] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:31:30.253474 22653 raft_consensus.cc:482] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:31:30.253531 22653 raft_consensus.cc:3037] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:31:30.254577 22653 raft_consensus.cc:504] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:30.254935 22653 leader_election.cc:288] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0 [CANDIDATE]: Term 1 election: Requested vote from peers 135b2386669b4430a003c3cf6dbb7728 (127.7.125.67:44891), 2bf980017525452096a42bbd2ef96c08 (127.7.125.65:43577)
I20260808 14:31:30.255332 22579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76512cfb0447438084827a10baa0f662" candidate_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "135b2386669b4430a003c3cf6dbb7728"
I20260808 14:31:30.255462 22579 raft_consensus.cc:3037] T 76512cfb0447438084827a10baa0f662 P 135b2386669b4430a003c3cf6dbb7728 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:31:30.255455 22293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76512cfb0447438084827a10baa0f662" candidate_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bf980017525452096a42bbd2ef96c08"
I20260808 14:31:30.255613 22293 raft_consensus.cc:3037] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:31:30.256692 22579 raft_consensus.cc:2454] T 76512cfb0447438084827a10baa0f662 P 135b2386669b4430a003c3cf6dbb7728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2714e1b2cf4ada877e1c1a992c61a0 in term 1.
I20260808 14:31:30.256839 22293 raft_consensus.cc:2454] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2714e1b2cf4ada877e1c1a992c61a0 in term 1.
I20260808 14:31:30.257022 22365 leader_election.cc:302] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 135b2386669b4430a003c3cf6dbb7728, 9c2714e1b2cf4ada877e1c1a992c61a0; no voters:
I20260808 14:31:30.257210 22653 raft_consensus.cc:2781] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:31:30.257491 22653 raft_consensus.cc:686] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2714e1b2cf4ada877e1c1a992c61a0, State: Running, Role: LEADER
I20260808 14:31:30.257860 22653 consensus_queue.cc:237] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:30.261196 22135 catalog_manager.cc:5495] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2714e1b2cf4ada877e1c1a992c61a0 (127.7.125.66). New cstate: current_term: 1 leader_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } health_report { overall_health: UNKNOWN } } }
I20260808 14:31:30.296573 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:30.293239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49184 (local address 127.7.125.66:42295)
0808 14:31:30.293375 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:30.293379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:30.293393 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:31:30.293482 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:30.293484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:30.293542 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:30.293637 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:31:30.293854 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.294348 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.295272 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.295486 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.295564 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:30.295823 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:31:30.296253 (+ 430us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:31:30.296343 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:30.296408 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 14:31:30.296458 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 14:31:30.302585 22579 raft_consensus.cc:1261] T 76512cfb0447438084827a10baa0f662 P 135b2386669b4430a003c3cf6dbb7728 [term 1 FOLLOWER]: Refusing update from remote peer 9c2714e1b2cf4ada877e1c1a992c61a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:31:30.302587 22293 raft_consensus.cc:1261] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Refusing update from remote peer 9c2714e1b2cf4ada877e1c1a992c61a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:31:30.303429 22653 consensus_queue.cc:1035] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:31:30.303694 22655 consensus_queue.cc:1035] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:31:30.311108 22660 mvcc.cc:205] Tried to move back new op lower bound from 7316273112271654912 to 7316273112099119104. Current Snapshot: MvccSnapshot[applied={T|T < 7316273112271654912}]
I20260808 14:31:30.312764 22659 mvcc.cc:205] Tried to move back new op lower bound from 7316273112271654912 to 7316273112099119104. Current Snapshot: MvccSnapshot[applied={T|T < 7316273112271654912}]
I20260808 14:31:30.321393 22135 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:31:30.321599 22135 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:31:30.324201 22135 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a: Sending DeleteTablet for 3 replicas of tablet 76512cfb0447438084827a10baa0f662
I20260808 14:31:30.325096 22559 tablet_service.cc:1514] Processing DeleteTablet for tablet 76512cfb0447438084827a10baa0f662 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:31:30 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:38386
I20260808 14:31:30.325188 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 22203
I20260808 14:31:30.325331 22418 tablet_service.cc:1514] Processing DeleteTablet for tablet 76512cfb0447438084827a10baa0f662 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:31:30 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:49172
I20260808 14:31:30.327463 22670 tablet_replica.cc:330] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0: stopping tablet replica
I20260808 14:31:30.327708 22669 tablet_replica.cc:330] T 76512cfb0447438084827a10baa0f662 P 135b2386669b4430a003c3cf6dbb7728: stopping tablet replica
I20260808 14:31:30.327962 22670 raft_consensus.cc:2229] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:31:30.327979 22669 raft_consensus.cc:2229] T 76512cfb0447438084827a10baa0f662 P 135b2386669b4430a003c3cf6dbb7728 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:31:30.328327 22669 raft_consensus.cc:2258] T 76512cfb0447438084827a10baa0f662 P 135b2386669b4430a003c3cf6dbb7728 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:31:30.328491 22670 raft_consensus.cc:2258] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:31:30.330616 22670 ts_tablet_manager.cc:1905] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:31:30.330616 22669 ts_tablet_manager.cc:1905] T 76512cfb0447438084827a10baa0f662 P 135b2386669b4430a003c3cf6dbb7728: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:31:30.333683 22670 ts_tablet_manager.cc:1918] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:31:30.333992 22670 log.cc:1196] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/76512cfb0447438084827a10baa0f662
I20260808 14:31:30.334216 22669 ts_tablet_manager.cc:1918] T 76512cfb0447438084827a10baa0f662 P 135b2386669b4430a003c3cf6dbb7728: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:31:30.334336 22669 log.cc:1196] T 76512cfb0447438084827a10baa0f662 P 135b2386669b4430a003c3cf6dbb7728: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/76512cfb0447438084827a10baa0f662
I20260808 14:31:30.334663 22670 ts_tablet_manager.cc:1939] T 76512cfb0447438084827a10baa0f662 P 9c2714e1b2cf4ada877e1c1a992c61a0: Deleting consensus metadata
I20260808 14:31:30.334718 22669 ts_tablet_manager.cc:1939] T 76512cfb0447438084827a10baa0f662 P 135b2386669b4430a003c3cf6dbb7728: Deleting consensus metadata
I20260808 14:31:30.336210 22121 catalog_manager.cc:4841] TS 135b2386669b4430a003c3cf6dbb7728 (127.7.125.67:44891): tablet 76512cfb0447438084827a10baa0f662 (table test-table [id=0c4f887c4749486aace5ec5ddf10f837]) successfully deleted
I20260808 14:31:30.336486 22121 catalog_manager.cc:4841] TS 9c2714e1b2cf4ada877e1c1a992c61a0 (127.7.125.66:42295): tablet 76512cfb0447438084827a10baa0f662 (table test-table [id=0c4f887c4749486aace5ec5ddf10f837]) successfully deleted
W20260808 14:31:30.336944 22121 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.125.65:43577 (error 108)
I20260808 14:31:30.337183 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 22348
I20260808 14:31:30.339345 22637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:30.339057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:43577 (local address 127.0.0.1:42700)
0808 14:31:30.339188 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:30.339266 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.65:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 14:31:30.339591 22121 catalog_manager.cc:4584] TS 2bf980017525452096a42bbd2ef96c08 (127.7.125.65:43577): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 76512cfb0447438084827a10baa0f662: Network error: Client connection negotiation failed: client connection to 127.7.125.65:43577: connect: Connection refused (error 111)
I20260808 14:31:30.344260 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 22491
I20260808 14:31:30.351608 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 22104
I20260808 14:31:30.360587 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:43375
--webserver_interface=127.7.125.126
--webserver_port=34677
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:43435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:43375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:31:30.473227 22672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:30.473616 22672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:30.473703 22672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:30.477800 22672 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:31:30.477914 22672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:30.477953 22672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:30.477995 22672 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:31:30.478034 22672 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:31:30.483531 22672 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:43435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:43375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:43375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=34677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:30.484710 22672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:30.485791 22672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:30.501082 22679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:30.501106 22678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:30.501446 22672 server_base.cc:1034] running on GCE node
W20260808 14:31:30.501142 22681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:30.502210 22672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:30.503572 22672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:30.504765 22672 hybrid_clock.cc:648] HybridClock initialized: now 1786199490504750 us; error 30 us; skew 500 ppm
Aug 08 14:31:30 dist-test-slave-w206 krb5kdc[22088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199490, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:30.507738 22672 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:31:30.508737 22672 webserver.cc:460] Webserver started at http://127.7.125.126:34677/ using document root <none> and password file <none>
I20260808 14:31:30.509344 22672 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:30.509469 22672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:30.513514 22672 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:31:30.515802 22688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:30.517069 22672 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 14:31:30.517217 22672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "17fe17e03d7a498897f8d082462f118a"
format_stamp: "Formatted at 2026-08-08 14:31:29 on dist-test-slave-w206"
server_key: "9b48449ff31d8f7a53ba7b76e3fc0a92"
server_key_iv: "55da9ad9f0a4fa8d8c532ae23fa2a9bf"
server_key_version: "encryptionkey@0"
I20260808 14:31:30.517880 22672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:30.530019 22672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:30.533085 22672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:30.533319 22672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:30.541028 22672 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:43375
I20260808 14:31:30.541347 22743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:43375 every 8 connection(s)
I20260808 14:31:30.542215 22672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:31:30.546684 22744 sys_catalog.cc:263] Verifying existing consensus state
I20260808 14:31:30.548487 22744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a: Bootstrap starting.
I20260808 14:31:30.548498 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 22672
I20260808 14:31:30.549012 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:43577
--local_ip_for_outbound_sockets=127.7.125.65
--tserver_master_addrs=127.7.125.126:43375
--webserver_port=32795
--webserver_interface=127.7.125.65
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:43435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:31:30.566532 22744 log.cc:824] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a: Log is configured to *not* fsync() on all Append() calls
I20260808 14:31:30.575433 22744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:31:30.575830 22744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a: Bootstrap complete.
I20260808 14:31:30.580274 22744 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fe17e03d7a498897f8d082462f118a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43375 } }
I20260808 14:31:30.580889 22744 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fe17e03d7a498897f8d082462f118a, State: Initialized, Role: FOLLOWER
I20260808 14:31:30.581472 22744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fe17e03d7a498897f8d082462f118a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43375 } }
I20260808 14:31:30.581641 22744 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:31:30.581719 22744 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:31:30.581822 22744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 1 FOLLOWER]: Advancing to term 2
I20260808 14:31:30.584259 22744 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fe17e03d7a498897f8d082462f118a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43375 } }
I20260808 14:31:30.584620 22744 leader_election.cc:302] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17fe17e03d7a498897f8d082462f118a; no voters:
I20260808 14:31:30.584952 22744 leader_election.cc:288] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 14:31:30.585099 22748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 2 FOLLOWER]: Leader election won for term 2
I20260808 14:31:30.585471 22748 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 2 LEADER]: Becoming Leader. State: Replica: 17fe17e03d7a498897f8d082462f118a, State: Running, Role: LEADER
I20260808 14:31:30.585803 22748 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fe17e03d7a498897f8d082462f118a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43375 } }
I20260808 14:31:30.586305 22744 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:31:30.587540 22750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17fe17e03d7a498897f8d082462f118a. Latest consensus state: current_term: 2 leader_uuid: "17fe17e03d7a498897f8d082462f118a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fe17e03d7a498897f8d082462f118a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43375 } } }
I20260808 14:31:30.587551 22749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "17fe17e03d7a498897f8d082462f118a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fe17e03d7a498897f8d082462f118a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43375 } } }
I20260808 14:31:30.587684 22750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [sys.catalog]: This master's current role is: LEADER
I20260808 14:31:30.587687 22749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [sys.catalog]: This master's current role is: LEADER
I20260808 14:31:30.588056 22757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:31:30.593115 22757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:31:30.593766 22757 catalog_manager.cc:1259] Loaded cluster ID: bb89ed74fe1d4d71973af55096723866
I20260808 14:31:30.593829 22757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:31:30.595247 22757 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 14:31:30.599994 22757 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 14:31:30.605512 22757 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:31:30.606299 22757 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a: Loaded TSK: 0
I20260808 14:31:30.607594 22757 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 14:31:30.674615 22746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:30.674897 22746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:30.674943 22746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:30.679338 22746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:30.679400 22746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:30.679466 22746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:31:30.684852 22746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:43435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:43577
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=32795
--tserver_master_addrs=127.7.125.126:43375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:30.685992 22746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:30.687264 22746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:30.697211 22773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:30.697214 22771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:30.697355 22775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:30.697423 22746 server_base.cc:1034] running on GCE node
I20260808 14:31:30.697883 22746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:30.698587 22746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:30.699770 22746 hybrid_clock.cc:648] HybridClock initialized: now 1786199490699718 us; error 70 us; skew 500 ppm
Aug 08 14:31:30 dist-test-slave-w206 krb5kdc[22088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199490, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:30.703433 22746 init.cc:377] Logged in from keytab as kudu/127.7.125.65@KRBTEST.COM (short username kudu)
I20260808 14:31:30.704624 22746 webserver.cc:460] Webserver started at http://127.7.125.65:32795/ using document root <none> and password file <none>
I20260808 14:31:30.705344 22746 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:30.705477 22746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:30.710882 22746 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:31:30.713873 22782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:30.715548 22746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 14:31:30.715684 22746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "2bf980017525452096a42bbd2ef96c08"
format_stamp: "Formatted at 2026-08-08 14:31:29 on dist-test-slave-w206"
server_key: "bdc5e36189de974eaa6bc4473d836da4"
server_key_iv: "aa9bfa2c378f6808fe240b0b56879196"
server_key_version: "encryptionkey@0"
I20260808 14:31:30.716426 22746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:30.736831 22746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:30.740653 22746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:30.740893 22746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:30.741595 22746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:31:30.742954 22789 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 14:31:30.747148 22746 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 14:31:30.747225 22746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260808 14:31:30.747293 22746 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 14:31:30.749190 22746 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 14:31:30.749236 22746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 14:31:30.749577 22789 tablet_bootstrap.cc:492] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08: Bootstrap starting.
I20260808 14:31:30.759799 22746 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:43577
I20260808 14:31:30.759872 22901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:43577 every 8 connection(s)
I20260808 14:31:30.760797 22746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:31:30.767532 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 22746
I20260808 14:31:30.768108 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:42295
--local_ip_for_outbound_sockets=127.7.125.66
--tserver_master_addrs=127.7.125.126:43375
--webserver_port=32937
--webserver_interface=127.7.125.66
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:43435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:31:30.769851 22789 log.cc:824] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:31:30 dist-test-slave-w206 krb5kdc[22088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199490, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:31:30.778014 22789 tablet_bootstrap.cc:492] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:31:30.778476 22789 tablet_bootstrap.cc:492] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08: Bootstrap complete.
I20260808 14:31:30.779968 22789 ts_tablet_manager.cc:1397] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08: Time spent bootstrapping tablet: real 0.031s user 0.015s sys 0.015s
I20260808 14:31:30.781195 22905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:30.762789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.65:49763)
0808 14:31:30.763724 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:30.763782 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:30.765071 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:30.766240 (+ 1169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:30.766255 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:30.766777 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:30.767581 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:30.767612 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.769588 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.769592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:30.770756 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:30.770766 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.771603 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.771607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:30.771809 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:30.772565 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:30.772591 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:30.774559 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:30.777455 (+ 2896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:30.777461 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:30.777473 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:30.777814 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:30.778249 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:30.778254 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:30.778257 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:30.778450 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:30.779360 (+ 910us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:30.779376 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:30.779674 (+ 298us) client_negotiation.cc:769] Sending connection context
0808 14:31:30.779875 (+ 201us) client_negotiation.cc:240] Negotiation successful
0808 14:31:30.780293 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"thread_start_us":201,"threads_started":1}
I20260808 14:31:30.781358 22904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:30.762877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:49763 (local address 127.7.125.126:43375)
0808 14:31:30.763382 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:30.763406 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:30.763926 (+ 520us) server_negotiation.cc:407] Connection header received
0808 14:31:30.765481 (+ 1555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:30.765491 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:30.765782 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:30.766059 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:30.767865 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.769402 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.770902 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.771445 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.771635 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:30.774785 (+ 3150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:30.774795 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:30.774815 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:30.774860 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:30.777269 (+ 2409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:30.777951 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:30.777960 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:30.777967 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:30.778135 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:30.778640 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:30.778645 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:30.778650 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:30.779214 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:30.779369 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:30.780059 (+ 690us) server_negotiation.cc:299] Negotiation successful
0808 14:31:30.780405 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":161,"threads_started":1}
I20260808 14:31:30.782835 22789 raft_consensus.cc:348] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:30.783377 22789 raft_consensus.cc:729] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bf980017525452096a42bbd2ef96c08, State: Initialized, Role: FOLLOWER
I20260808 14:31:30.783888 22789 consensus_queue.cc:260] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:30.784188 22902 heartbeater.cc:347] Connected to a master server at 127.7.125.126:43375
I20260808 14:31:30.784418 22902 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:30.784943 22902 heartbeater.cc:511] Master 127.7.125.126:43375 requested a full tablet report, sending...
I20260808 14:31:30.784940 22789 ts_tablet_manager.cc:1428] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260808 14:31:30.787549 22707 ts_manager.cc:194] Registered new tserver with Master: 2bf980017525452096a42bbd2ef96c08 (127.7.125.65:43577)
I20260808 14:31:30.791494 22707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:49763
I20260808 14:31:30.794394 22902 heartbeater.cc:503] Master 127.7.125.126:43375 was elected leader, sending a full tablet report...
Aug 08 14:31:30 dist-test-slave-w206 krb5kdc[22088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199490, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for kudu/127.7.125.65@KRBTEST.COM
I20260808 14:31:30.800967 22909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:30.790815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:43577 (local address 127.0.0.1:42714)
0808 14:31:30.791230 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:30.791273 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:30.791410 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:30.792055 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:30.792061 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:30.792421 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:30.792521 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:30.792527 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.793647 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.793653 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:30.794261 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:30.794270 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.795061 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.795064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:30.795184 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:30.795745 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:30.795756 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:30.797299 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:30.799365 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:30.799375 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:30.799380 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:30.799602 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:30.799906 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:30.799908 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:30.799909 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:30.799992 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:30.800479 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:30.800483 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:30.800577 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 14:31:30.800684 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 14:31:30.800827 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":157,"threads_started":1}
I20260808 14:31:30.801110 22910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:30.790864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42714 (local address 127.7.125.65:43577)
0808 14:31:30.791639 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:30.791651 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:30.791673 (+ 22us) server_negotiation.cc:407] Connection header received
0808 14:31:30.791744 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:30.791748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:30.791904 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:30.792053 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:30.792646 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.793536 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.794384 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.794885 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.795047 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:30.797446 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:30.797456 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:30.797469 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:30.797496 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:30.799260 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:30.799712 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:30.799718 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:30.799720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:30.799810 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:30.800075 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:30.800078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:30.800080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:30.800370 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:30.800499 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:30.800786 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 14:31:30.800957 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":122,"threads_started":1}
I20260808 14:31:30.801924 22832 tablet_service.cc:1514] Processing DeleteTablet for tablet 76512cfb0447438084827a10baa0f662 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:31:30 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:42714
I20260808 14:31:30.802745 22911 tablet_replica.cc:330] stopping tablet replica
I20260808 14:31:30.803117 22911 raft_consensus.cc:2229] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:31:30.803364 22911 raft_consensus.cc:2258] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:31:30.804337 22911 ts_tablet_manager.cc:1905] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:31:30.807891 22911 ts_tablet_manager.cc:1918] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:31:30.807973 22911 log.cc:1196] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/wal/wals/76512cfb0447438084827a10baa0f662
I20260808 14:31:30.808254 22911 ts_tablet_manager.cc:1939] T 76512cfb0447438084827a10baa0f662 P 2bf980017525452096a42bbd2ef96c08: Deleting consensus metadata
I20260808 14:31:30.809981 22690 catalog_manager.cc:4841] TS 2bf980017525452096a42bbd2ef96c08 (127.7.125.65:43577): tablet 76512cfb0447438084827a10baa0f662 (table test-table [id=0c4f887c4749486aace5ec5ddf10f837]) successfully deleted
W20260808 14:31:30.891987 22906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:30.892282 22906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:30.892364 22906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:30.896462 22906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:30.896554 22906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:30.896646 22906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:31:30.902084 22906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:43435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:42295
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=32937
--tserver_master_addrs=127.7.125.126:43375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:30.903218 22906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:30.904249 22906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:30.914271 22920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:30.914523 22906 server_base.cc:1034] running on GCE node
W20260808 14:31:30.914441 22916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:30.914271 22918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:30.915050 22906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:30.915729 22906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:30.916916 22906 hybrid_clock.cc:648] HybridClock initialized: now 1786199490916889 us; error 24 us; skew 500 ppm
Aug 08 14:31:30 dist-test-slave-w206 krb5kdc[22088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199490, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:30.919703 22906 init.cc:377] Logged in from keytab as kudu/127.7.125.66@KRBTEST.COM (short username kudu)
I20260808 14:31:30.920554 22906 webserver.cc:460] Webserver started at http://127.7.125.66:32937/ using document root <none> and password file <none>
I20260808 14:31:30.921056 22906 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:30.921129 22906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:30.925079 22906 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 14:31:30.927103 22927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:30.928278 22906 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:31:30.928411 22906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "9c2714e1b2cf4ada877e1c1a992c61a0"
format_stamp: "Formatted at 2026-08-08 14:31:29 on dist-test-slave-w206"
server_key: "941c2c7c1969bfbc408b56d3949c96da"
server_key_iv: "e2d3a92a2a2d5621794d8117060f591c"
server_key_version: "encryptionkey@0"
I20260808 14:31:30.928910 22906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:30.946768 22906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:30.950318 22906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:30.950546 22906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:30.951201 22906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:31:30.952045 22906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:31:30.952093 22906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:30.952165 22906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:31:30.952200 22906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:30.961427 22906 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:42295
I20260808 14:31:30.961457 23046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:42295 every 8 connection(s)
I20260808 14:31:30.962483 22906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:31:30.968024 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 22906
I20260808 14:31:30.968459 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:44891
--local_ip_for_outbound_sockets=127.7.125.67
--tserver_master_addrs=127.7.125.126:43375
--webserver_port=39557
--webserver_interface=127.7.125.67
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:43435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 14:31:30 dist-test-slave-w206 krb5kdc[22088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199490, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:31:30.976365 22904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:30.964404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:54989 (local address 127.7.125.126:43375)
0808 14:31:30.964555 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:30.964560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:30.965351 (+ 791us) server_negotiation.cc:407] Connection header received
0808 14:31:30.966692 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:30.966695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:30.966758 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:30.966836 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:30.968075 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.968730 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.969905 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.970157 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.970275 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:30.972780 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:30.972784 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:30.972787 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:30.972821 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:30.974281 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:30.974802 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:30.974807 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:30.974814 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:30.974895 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:30.975264 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:30.975267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:30.975269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:30.975448 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:30.975557 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:30.976043 (+ 486us) server_negotiation.cc:299] Negotiation successful
0808 14:31:30.976174 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:31:30.977176 23049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:30.964738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.66:54989)
0808 14:31:30.965190 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:30.965235 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:30.966443 (+ 1208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:30.967035 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:30.967042 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:30.967438 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:30.967914 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:30.967928 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.968882 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.968886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:30.969768 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:30.969777 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:30.970318 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:30.970322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:30.970499 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:30.971147 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:30.971163 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:30.972612 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:30.974420 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:30.974426 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:30.974440 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:30.974692 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:30.975010 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:30.975013 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:30.975015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:30.975158 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:30.975556 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:30.975562 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:30.975798 (+ 236us) client_negotiation.cc:769] Sending connection context
0808 14:31:30.975988 (+ 190us) client_negotiation.cc:240] Negotiation successful
0808 14:31:30.976318 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":115,"threads_started":1}
I20260808 14:31:30.978498 23047 heartbeater.cc:347] Connected to a master server at 127.7.125.126:43375
I20260808 14:31:30.978801 23047 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:30.979385 23047 heartbeater.cc:511] Master 127.7.125.126:43375 requested a full tablet report, sending...
I20260808 14:31:30.980525 22707 ts_manager.cc:194] Registered new tserver with Master: 9c2714e1b2cf4ada877e1c1a992c61a0 (127.7.125.66:42295)
I20260808 14:31:30.981277 22707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:54989
W20260808 14:31:31.082731 23050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:31.083073 23050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:31.083125 23050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:31.087262 23050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:31.087319 23050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:31.087379 23050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:31:31.092818 23050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:43435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:44891
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=39557
--tserver_master_addrs=127.7.125.126:43375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:31.093999 23050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:31.095127 23050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:31.104805 23055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:31.104874 23050 server_base.cc:1034] running on GCE node
W20260808 14:31:31.104805 23056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:31.104805 23058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:31.105505 23050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:31.106233 23050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:31.107410 23050 hybrid_clock.cc:648] HybridClock initialized: now 1786199491107380 us; error 47 us; skew 500 ppm
Aug 08 14:31:31 dist-test-slave-w206 krb5kdc[22088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199491, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:31.110213 23050 init.cc:377] Logged in from keytab as kudu/127.7.125.67@KRBTEST.COM (short username kudu)
I20260808 14:31:31.111225 23050 webserver.cc:460] Webserver started at http://127.7.125.67:39557/ using document root <none> and password file <none>
I20260808 14:31:31.111796 23050 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:31.111922 23050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:31.116071 23050 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:31:31.118189 23065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:31.119500 23050 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:31:31.119642 23050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "135b2386669b4430a003c3cf6dbb7728"
format_stamp: "Formatted at 2026-08-08 14:31:30 on dist-test-slave-w206"
server_key: "b21110fd0531d3fa3825ca7841d782dd"
server_key_iv: "1c8f7a1c7540a29ca6f1d58fade22d8e"
server_key_version: "encryptionkey@0"
I20260808 14:31:31.120200 23050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:31.139433 23050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:31.142513 23050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:31.142740 23050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:31.143468 23050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:31:31.144353 23050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:31:31.144431 23050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:31.144505 23050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:31:31.144558 23050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:31.153621 23050 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:44891
I20260808 14:31:31.153650 23187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:44891 every 8 connection(s)
I20260808 14:31:31.154722 23050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:31:31.156179 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 23050
Aug 08 14:31:31 dist-test-slave-w206 krb5kdc[22088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199491, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:31:31.166091 23192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:31.156907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56456 (local address 127.7.125.126:43375)
0808 14:31:31.157254 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:31.157257 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:31.157327 (+ 70us) server_negotiation.cc:407] Connection header received
0808 14:31:31.157486 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:31.157489 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:31.157532 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:31.157663 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:31.158623 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.159497 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.160463 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.160792 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.160909 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:31.162439 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:31.162443 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:31.162446 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:31.162476 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:31.164326 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:31.164781 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:31.164787 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:31.164794 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:31.164879 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:31.165175 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:31.165179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:31.165180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:31.165361 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:31.165580 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:31.165767 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 14:31:31.165911 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":124,"server-negotiator.queue_time_us":288,"thread_start_us":98,"threads_started":1}
I20260808 14:31:31.170053 22904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:31.156896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:56969 (local address 127.7.125.126:43375)
0808 14:31:31.157017 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:31.157021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:31.158024 (+ 1003us) server_negotiation.cc:407] Connection header received
0808 14:31:31.159626 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:31.159630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:31.159689 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:31.159776 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:31.161190 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.161912 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.163527 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.163819 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.163911 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:31.166554 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:31.166564 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:31.166567 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:31.166587 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:31.168079 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:31.168579 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:31.168584 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:31.168586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:31.168665 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:31.169015 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:31.169019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:31.169021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:31.169190 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:31.169296 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:31.169794 (+ 498us) server_negotiation.cc:299] Negotiation successful
0808 14:31:31.169904 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 14:31:31.170928 23194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:31.157323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.67:56969)
0808 14:31:31.157875 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:31.157926 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:31.159376 (+ 1450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:31.159975 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:31.159982 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:31.160336 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:31.161034 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:31.161057 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.162053 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.162056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:31.163395 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:31.163410 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.163987 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.163991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:31.164223 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:31.164815 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:31.164832 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:31.166390 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:31.168193 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:31.168199 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:31.168214 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:31.168475 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:31.168759 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:31.168763 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:31.168765 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:31.168917 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:31.169296 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:31.169304 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:31.169560 (+ 256us) client_negotiation.cc:769] Sending connection context
0808 14:31:31.169754 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 14:31:31.170102 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":127,"threads_started":1}
I20260808 14:31:31.172051 23188 heartbeater.cc:347] Connected to a master server at 127.7.125.126:43375
I20260808 14:31:31.172294 23188 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:31.172807 23188 heartbeater.cc:511] Master 127.7.125.126:43375 requested a full tablet report, sending...
I20260808 14:31:31.173807 22707 ts_manager.cc:194] Registered new tserver with Master: 135b2386669b4430a003c3cf6dbb7728 (127.7.125.67:44891)
I20260808 14:31:31.174372 22707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:56969
I20260808 14:31:31.179780 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:31:31.183853 22904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:31.180373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56460 (local address 127.7.125.126:43375)
0808 14:31:31.180523 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:31.180528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:31.180707 (+ 179us) server_negotiation.cc:407] Connection header received
0808 14:31:31.180826 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:31.180828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:31.180933 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:31.181019 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:31:31.181229 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.181719 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.182537 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.182735 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.182806 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:31.183120 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:31:31.183506 (+ 386us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:31:31.183594 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:31.183656 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 14:31:31.183699 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 14:31:31.184506 22707 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:56460:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:31:31.188040 22707 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:31:31.196136 22832 tablet_service.cc:1467] Processing CreateTablet for tablet f8be0d6da9be4a2fa126b5b52f7d6783 (DEFAULT_TABLE table=test-table [id=f61797cfb5b04bb9aaa98818baf255ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:31:31.196751 22832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8be0d6da9be4a2fa126b5b52f7d6783. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:31:31.199826 22789 tablet_bootstrap.cc:492] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08: Bootstrap starting.
I20260808 14:31:31.200721 22789 tablet_bootstrap.cc:654] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08: Neither blocks nor log segments found. Creating new log.
I20260808 14:31:31.200749 22909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:31.194766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:44891 (local address 127.0.0.1:38400)
0808 14:31:31.194908 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:31.194925 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:31.195026 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:31.196015 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:31.196018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:31.196037 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:31:31.196148 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:31.196158 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.197823 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.197825 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:31.199046 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:31.199057 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.200305 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.200308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:31.200409 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:31.200452 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 14:31:31.200530 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 14:31:31.200621 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 14:31:31.200802 23200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:31.195148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38400 (local address 127.7.125.67:44891)
0808 14:31:31.195463 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:31.195480 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:31.195505 (+ 25us) server_negotiation.cc:407] Connection header received
0808 14:31:31.195587 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:31.195592 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:31.195825 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:31.196036 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:31:31.196284 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.197666 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.199182 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.200181 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.200305 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:31.200457 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:31.200576 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 14:31:31.200668 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":127,"threads_started":1}
I20260808 14:31:31.201303 23199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:31.195118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:42295 (local address 127.0.0.1:49192)
0808 14:31:31.195361 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:31.195374 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:31.195462 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:31.197561 (+ 2099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:31.197564 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:31.197581 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:31:31.197672 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:31.197678 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.198659 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.198664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:31.199904 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:31.199930 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.201022 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.201025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:31.201092 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:31.201111 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 14:31:31.201158 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 14:31:31.201206 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":114,"threads_started":1}
I20260808 14:31:31.201503 23201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:31.195915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49192 (local address 127.7.125.66:42295)
0808 14:31:31.197117 (+ 1202us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:31.197129 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:31.197152 (+ 23us) server_negotiation.cc:407] Connection header received
0808 14:31:31.197226 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:31.197231 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:31.197421 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:31.197568 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:31:31.197778 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.198521 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.200049 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.200882 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.201006 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:31.201182 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:31.201238 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 14:31:31.201326 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1097,"thread_start_us":131,"threads_started":1}
I20260808 14:31:31.202394 22789 tablet_bootstrap.cc:492] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08: No bootstrap required, opened a new log
I20260808 14:31:31.202520 22789 ts_tablet_manager.cc:1397] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260808 14:31:31.202912 22789 raft_consensus.cc:348] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:31.203061 22789 raft_consensus.cc:374] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:31:31.203109 22789 raft_consensus.cc:729] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bf980017525452096a42bbd2ef96c08, State: Initialized, Role: FOLLOWER
I20260808 14:31:31.203245 22789 consensus_queue.cc:260] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:31.203510 22789 ts_tablet_manager.cc:1428] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:31:31.205116 23117 tablet_service.cc:1467] Processing CreateTablet for tablet f8be0d6da9be4a2fa126b5b52f7d6783 (DEFAULT_TABLE table=test-table [id=f61797cfb5b04bb9aaa98818baf255ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:31:31.205214 22977 tablet_service.cc:1467] Processing CreateTablet for tablet f8be0d6da9be4a2fa126b5b52f7d6783 (DEFAULT_TABLE table=test-table [id=f61797cfb5b04bb9aaa98818baf255ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:31:31.206158 23117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8be0d6da9be4a2fa126b5b52f7d6783. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:31:31.206234 22977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8be0d6da9be4a2fa126b5b52f7d6783. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:31:31.212920 23204 tablet_bootstrap.cc:492] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0: Bootstrap starting.
I20260808 14:31:31.212934 23205 tablet_bootstrap.cc:492] T f8be0d6da9be4a2fa126b5b52f7d6783 P 135b2386669b4430a003c3cf6dbb7728: Bootstrap starting.
I20260808 14:31:31.215430 23205 tablet_bootstrap.cc:654] T f8be0d6da9be4a2fa126b5b52f7d6783 P 135b2386669b4430a003c3cf6dbb7728: Neither blocks nor log segments found. Creating new log.
I20260808 14:31:31.215778 23204 tablet_bootstrap.cc:654] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0: Neither blocks nor log segments found. Creating new log.
I20260808 14:31:31.216511 23204 log.cc:824] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0: Log is configured to *not* fsync() on all Append() calls
I20260808 14:31:31.216511 23205 log.cc:824] T f8be0d6da9be4a2fa126b5b52f7d6783 P 135b2386669b4430a003c3cf6dbb7728: Log is configured to *not* fsync() on all Append() calls
I20260808 14:31:31.218418 23204 tablet_bootstrap.cc:492] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0: No bootstrap required, opened a new log
I20260808 14:31:31.218429 23205 tablet_bootstrap.cc:492] T f8be0d6da9be4a2fa126b5b52f7d6783 P 135b2386669b4430a003c3cf6dbb7728: No bootstrap required, opened a new log
I20260808 14:31:31.218632 23204 ts_tablet_manager.cc:1397] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:31:31.218638 23205 ts_tablet_manager.cc:1397] T f8be0d6da9be4a2fa126b5b52f7d6783 P 135b2386669b4430a003c3cf6dbb7728: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:31:31.222019 23204 raft_consensus.cc:348] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:31.222018 23205 raft_consensus.cc:348] T f8be0d6da9be4a2fa126b5b52f7d6783 P 135b2386669b4430a003c3cf6dbb7728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:31.222286 23204 raft_consensus.cc:374] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:31:31.222285 23205 raft_consensus.cc:374] T f8be0d6da9be4a2fa126b5b52f7d6783 P 135b2386669b4430a003c3cf6dbb7728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:31:31.222347 23204 raft_consensus.cc:729] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2714e1b2cf4ada877e1c1a992c61a0, State: Initialized, Role: FOLLOWER
I20260808 14:31:31.222357 23205 raft_consensus.cc:729] T f8be0d6da9be4a2fa126b5b52f7d6783 P 135b2386669b4430a003c3cf6dbb7728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 135b2386669b4430a003c3cf6dbb7728, State: Initialized, Role: FOLLOWER
I20260808 14:31:31.222823 23205 consensus_queue.cc:260] T f8be0d6da9be4a2fa126b5b52f7d6783 P 135b2386669b4430a003c3cf6dbb7728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:31.222797 23204 consensus_queue.cc:260] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:31.223510 23047 heartbeater.cc:503] Master 127.7.125.126:43375 was elected leader, sending a full tablet report...
I20260808 14:31:31.223843 23204 ts_tablet_manager.cc:1428] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 14:31:31.223857 23205 ts_tablet_manager.cc:1428] T f8be0d6da9be4a2fa126b5b52f7d6783 P 135b2386669b4430a003c3cf6dbb7728: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 14:31:31.224153 23188 heartbeater.cc:503] Master 127.7.125.126:43375 was elected leader, sending a full tablet report...
W20260808 14:31:31.262252 22903 tablet.cc:2369] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:31:31.343835 23208 raft_consensus.cc:482] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:31:31.344056 23208 raft_consensus.cc:504] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:31.345242 23208 leader_election.cc:288] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 135b2386669b4430a003c3cf6dbb7728 (127.7.125.67:44891), 2bf980017525452096a42bbd2ef96c08 (127.7.125.65:43577)
I20260808 14:31:31.349623 23200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:31.345634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:39295 (local address 127.7.125.67:44891)
0808 14:31:31.345804 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:31.345810 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:31.345832 (+ 22us) server_negotiation.cc:407] Connection header received
0808 14:31:31.345919 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:31.345923 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:31.345975 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:31.346121 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:31:31.346461 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.347208 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.348483 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.349107 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.349198 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:31.349240 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:31.349461 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 14:31:31.349523 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 14:31:31.349646 23049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:31.345504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:44891 (local address 127.7.125.66:39295)
0808 14:31:31.345663 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:31.345680 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:31.345784 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:31.346142 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:31.346145 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:31.346178 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:31:31.346307 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:31.346315 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.347317 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.347321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:31.348344 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:31.348353 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.349223 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.349227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:31.349330 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:31.349355 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 14:31:31.349435 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 14:31:31.349506 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":19}
I20260808 14:31:31.350354 23211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:31.346035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:44911 (local address 127.7.125.65:43577)
0808 14:31:31.346418 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:31.346425 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:31.346440 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:31:31.346568 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:31.346570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:31.346624 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:31.346710 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:31:31.347073 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.347791 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.349057 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.349729 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.349861 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:31.350041 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:31.350190 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 14:31:31.350255 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":97,"threads_started":1}
I20260808 14:31:31.350368 23210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:31.345838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:43577 (local address 127.7.125.66:44911)
0808 14:31:31.346335 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:31.346346 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:31.346461 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:31.346754 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:31.346757 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:31.346793 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:31:31.346924 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:31.346930 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.347915 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.347918 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:31.348919 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:31.348930 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.349922 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.349925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:31.350031 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:31.350080 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 14:31:31.350165 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 14:31:31.350236 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":19,"thread_start_us":161,"threads_started":1}
I20260808 14:31:31.350410 23137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8be0d6da9be4a2fa126b5b52f7d6783" candidate_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "135b2386669b4430a003c3cf6dbb7728" is_pre_election: true
I20260808 14:31:31.350745 23137 raft_consensus.cc:2454] T f8be0d6da9be4a2fa126b5b52f7d6783 P 135b2386669b4430a003c3cf6dbb7728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2714e1b2cf4ada877e1c1a992c61a0 in term 0.
I20260808 14:31:31.351104 22855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8be0d6da9be4a2fa126b5b52f7d6783" candidate_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bf980017525452096a42bbd2ef96c08" is_pre_election: true
I20260808 14:31:31.351250 22929 leader_election.cc:302] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 135b2386669b4430a003c3cf6dbb7728, 9c2714e1b2cf4ada877e1c1a992c61a0; no voters:
I20260808 14:31:31.351440 22855 raft_consensus.cc:2454] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2714e1b2cf4ada877e1c1a992c61a0 in term 0.
I20260808 14:31:31.351585 23208 raft_consensus.cc:2781] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:31:31.351701 23208 raft_consensus.cc:482] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:31:31.351771 23208 raft_consensus.cc:3037] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:31:31.352993 23208 raft_consensus.cc:504] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:31.353397 23208 leader_election.cc:288] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0 [CANDIDATE]: Term 1 election: Requested vote from peers 135b2386669b4430a003c3cf6dbb7728 (127.7.125.67:44891), 2bf980017525452096a42bbd2ef96c08 (127.7.125.65:43577)
I20260808 14:31:31.353780 22855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8be0d6da9be4a2fa126b5b52f7d6783" candidate_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bf980017525452096a42bbd2ef96c08"
I20260808 14:31:31.353782 23137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8be0d6da9be4a2fa126b5b52f7d6783" candidate_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "135b2386669b4430a003c3cf6dbb7728"
I20260808 14:31:31.353932 22855 raft_consensus.cc:3037] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:31:31.353938 23137 raft_consensus.cc:3037] T f8be0d6da9be4a2fa126b5b52f7d6783 P 135b2386669b4430a003c3cf6dbb7728 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:31:31.354949 23137 raft_consensus.cc:2454] T f8be0d6da9be4a2fa126b5b52f7d6783 P 135b2386669b4430a003c3cf6dbb7728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2714e1b2cf4ada877e1c1a992c61a0 in term 1.
I20260808 14:31:31.354954 22855 raft_consensus.cc:2454] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2714e1b2cf4ada877e1c1a992c61a0 in term 1.
I20260808 14:31:31.355265 22929 leader_election.cc:302] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bf980017525452096a42bbd2ef96c08, 9c2714e1b2cf4ada877e1c1a992c61a0; no voters:
I20260808 14:31:31.355460 23208 raft_consensus.cc:2781] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:31:31.355703 23208 raft_consensus.cc:686] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2714e1b2cf4ada877e1c1a992c61a0, State: Running, Role: LEADER
I20260808 14:31:31.356065 23208 consensus_queue.cc:237] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:31.359207 22706 catalog_manager.cc:5495] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2714e1b2cf4ada877e1c1a992c61a0 (127.7.125.66). New cstate: current_term: 1 leader_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } health_report { overall_health: UNKNOWN } } }
I20260808 14:31:31.397989 23201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:31.394731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49194 (local address 127.7.125.66:42295)
0808 14:31:31.394874 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:31.394877 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:31.394892 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:31:31.395036 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:31.395038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:31.395096 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:31.395187 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:31:31.395407 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.395890 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.396661 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:31.396864 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:31.396938 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:31.397194 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:31:31.397625 (+ 431us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:31:31.397713 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:31.397761 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 14:31:31.397812 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 14:31:31.404068 22855 raft_consensus.cc:1261] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Refusing update from remote peer 9c2714e1b2cf4ada877e1c1a992c61a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:31:31.404072 23137 raft_consensus.cc:1261] T f8be0d6da9be4a2fa126b5b52f7d6783 P 135b2386669b4430a003c3cf6dbb7728 [term 1 FOLLOWER]: Refusing update from remote peer 9c2714e1b2cf4ada877e1c1a992c61a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:31:31.404800 23212 consensus_queue.cc:1035] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:31:31.404989 23208 consensus_queue.cc:1035] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 14:31:31.405789 23189 tablet.cc:2369] T f8be0d6da9be4a2fa126b5b52f7d6783 P 135b2386669b4430a003c3cf6dbb7728: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:31:31.411752 23215 mvcc.cc:205] Tried to move back new op lower bound from 7316273116783304704 to 7316273116597096448. Current Snapshot: MvccSnapshot[applied={T|T < 7316273116783304704}]
I20260808 14:31:31.415092 23214 mvcc.cc:205] Tried to move back new op lower bound from 7316273116783304704 to 7316273116597096448. Current Snapshot: MvccSnapshot[applied={T|T < 7316273116783304704 or (T in {7316273116783304704})}]
I20260808 14:31:31.415581 23216 mvcc.cc:205] Tried to move back new op lower bound from 7316273116783304704 to 7316273116597096448. Current Snapshot: MvccSnapshot[applied={T|T < 7316273116783304704}]
I20260808 14:31:31.419932 22707 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:56460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:31:31.420109 22707 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:56460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:31:31.422708 22707 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a: Sending DeleteTablet for 3 replicas of tablet f8be0d6da9be4a2fa126b5b52f7d6783
I20260808 14:31:31.423571 22832 tablet_service.cc:1514] Processing DeleteTablet for tablet f8be0d6da9be4a2fa126b5b52f7d6783 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:31:31 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:42714
I20260808 14:31:31.423539 22977 tablet_service.cc:1514] Processing DeleteTablet for tablet f8be0d6da9be4a2fa126b5b52f7d6783 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:31:31 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:49192
I20260808 14:31:31.423620 23117 tablet_service.cc:1514] Processing DeleteTablet for tablet f8be0d6da9be4a2fa126b5b52f7d6783 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:31:31 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:38400
I20260808 14:31:31.424121 23224 tablet_replica.cc:330] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08: stopping tablet replica
I20260808 14:31:31.424122 23225 tablet_replica.cc:330] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0: stopping tablet replica
I20260808 14:31:31.424294 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 22746
I20260808 14:31:31.424295 23224 raft_consensus.cc:2229] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:31:31.424455 23224 raft_consensus.cc:2258] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:31:31.424568 23225 raft_consensus.cc:2229] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:31:31.424980 23225 raft_consensus.cc:2258] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:31:31.425387 23226 tablet_replica.cc:330] T f8be0d6da9be4a2fa126b5b52f7d6783 P 135b2386669b4430a003c3cf6dbb7728: stopping tablet replica
I20260808 14:31:31.425639 23226 raft_consensus.cc:2229] T f8be0d6da9be4a2fa126b5b52f7d6783 P 135b2386669b4430a003c3cf6dbb7728 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:31:31.425932 23226 raft_consensus.cc:2258] T f8be0d6da9be4a2fa126b5b52f7d6783 P 135b2386669b4430a003c3cf6dbb7728 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:31:31.427465 23225 ts_tablet_manager.cc:1905] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:31:31.431210 23225 ts_tablet_manager.cc:1918] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:31:31.431329 23225 log.cc:1196] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/f8be0d6da9be4a2fa126b5b52f7d6783
I20260808 14:31:31.431690 23225 ts_tablet_manager.cc:1939] T f8be0d6da9be4a2fa126b5b52f7d6783 P 9c2714e1b2cf4ada877e1c1a992c61a0: Deleting consensus metadata
I20260808 14:31:31.433151 22690 catalog_manager.cc:4841] TS 9c2714e1b2cf4ada877e1c1a992c61a0 (127.7.125.66:42295): tablet f8be0d6da9be4a2fa126b5b52f7d6783 (table test-table [id=f61797cfb5b04bb9aaa98818baf255ef]) successfully deleted
I20260808 14:31:31.435448 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 22906
I20260808 14:31:31.436241 23226 ts_tablet_manager.cc:1905] T f8be0d6da9be4a2fa126b5b52f7d6783 P 135b2386669b4430a003c3cf6dbb7728: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 14:31:31.437321 22690 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.125.65:43577 (error 108)
I20260808 14:31:31.438544 23199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:31.438207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:43577 (local address 127.0.0.1:42718)
0808 14:31:31.438361 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:31.438441 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.65:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 14:31:31.438735 22690 catalog_manager.cc:4584] TS 2bf980017525452096a42bbd2ef96c08 (127.7.125.65:43577): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f8be0d6da9be4a2fa126b5b52f7d6783: Network error: Client connection negotiation failed: client connection to 127.7.125.65:43577: connect: Connection refused (error 111)
I20260808 14:31:31.439502 23226 ts_tablet_manager.cc:1918] T f8be0d6da9be4a2fa126b5b52f7d6783 P 135b2386669b4430a003c3cf6dbb7728: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:31:31.439651 23226 log.cc:1196] T f8be0d6da9be4a2fa126b5b52f7d6783 P 135b2386669b4430a003c3cf6dbb7728: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/f8be0d6da9be4a2fa126b5b52f7d6783
I20260808 14:31:31.440317 23226 ts_tablet_manager.cc:1939] T f8be0d6da9be4a2fa126b5b52f7d6783 P 135b2386669b4430a003c3cf6dbb7728: Deleting consensus metadata
I20260808 14:31:31.441620 22690 catalog_manager.cc:4841] TS 135b2386669b4430a003c3cf6dbb7728 (127.7.125.67:44891): tablet f8be0d6da9be4a2fa126b5b52f7d6783 (table test-table [id=f61797cfb5b04bb9aaa98818baf255ef]) successfully deleted
I20260808 14:31:31.443989 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 23050
I20260808 14:31:31.450893 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 22672
I20260808 14:31:31.459348 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:43375
--webserver_interface=127.7.125.126
--webserver_port=34677
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:43435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:43375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:31:31.576817 23228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:31.577111 23228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:31.577219 23228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:31.581467 23228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:31:31.581548 23228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:31.581568 23228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:31.581586 23228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:31:31.581604 23228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:31:31.588788 23228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:43435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:43375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:43375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=34677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:31.590005 23228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:31.591169 23228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:31.600176 23237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:31.600176 23233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:31.600193 23234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:31.600456 23228 server_base.cc:1034] running on GCE node
I20260808 14:31:31.601238 23228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:31.602502 23228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:31.603695 23228 hybrid_clock.cc:648] HybridClock initialized: now 1786199491603683 us; error 29 us; skew 500 ppm
Aug 08 14:31:31 dist-test-slave-w206 krb5kdc[22088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199491, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:31.606540 23228 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:31:31.607558 23228 webserver.cc:460] Webserver started at http://127.7.125.126:34677/ using document root <none> and password file <none>
I20260808 14:31:31.608160 23228 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:31.608254 23228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:31.612401 23228 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:31:31.614821 23244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:31.616183 23228 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:31:31.616348 23228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "17fe17e03d7a498897f8d082462f118a"
format_stamp: "Formatted at 2026-08-08 14:31:29 on dist-test-slave-w206"
server_key: "9b48449ff31d8f7a53ba7b76e3fc0a92"
server_key_iv: "55da9ad9f0a4fa8d8c532ae23fa2a9bf"
server_key_version: "encryptionkey@0"
I20260808 14:31:31.616999 23228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:31.642676 23228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:31.645812 23228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:31.646046 23228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:31.653609 23228 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:43375
I20260808 14:31:31.653908 23302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:43375 every 8 connection(s)
I20260808 14:31:31.654799 23228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:31:31.657315 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 23228
I20260808 14:31:31.657848 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:43577
--local_ip_for_outbound_sockets=127.7.125.65
--tserver_master_addrs=127.7.125.126:43375
--webserver_port=32795
--webserver_interface=127.7.125.65
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:43435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:31:31.659441 23303 sys_catalog.cc:263] Verifying existing consensus state
I20260808 14:31:31.661116 23303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a: Bootstrap starting.
I20260808 14:31:31.679550 23303 log.cc:824] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a: Log is configured to *not* fsync() on all Append() calls
I20260808 14:31:31.690619 23303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:31:31.691095 23303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a: Bootstrap complete.
I20260808 14:31:31.695266 23303 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fe17e03d7a498897f8d082462f118a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43375 } }
I20260808 14:31:31.695703 23303 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fe17e03d7a498897f8d082462f118a, State: Initialized, Role: FOLLOWER
I20260808 14:31:31.696235 23303 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fe17e03d7a498897f8d082462f118a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43375 } }
I20260808 14:31:31.696380 23303 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:31:31.696475 23303 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:31:31.696592 23303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 2 FOLLOWER]: Advancing to term 3
I20260808 14:31:31.698500 23303 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fe17e03d7a498897f8d082462f118a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43375 } }
I20260808 14:31:31.698833 23303 leader_election.cc:302] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17fe17e03d7a498897f8d082462f118a; no voters:
I20260808 14:31:31.699218 23303 leader_election.cc:288] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 14:31:31.699311 23307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 3 FOLLOWER]: Leader election won for term 3
I20260808 14:31:31.699584 23307 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [term 3 LEADER]: Becoming Leader. State: Replica: 17fe17e03d7a498897f8d082462f118a, State: Running, Role: LEADER
I20260808 14:31:31.699877 23307 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fe17e03d7a498897f8d082462f118a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43375 } }
I20260808 14:31:31.700543 23303 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:31:31.701210 23308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "17fe17e03d7a498897f8d082462f118a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fe17e03d7a498897f8d082462f118a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43375 } } }
I20260808 14:31:31.701345 23308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [sys.catalog]: This master's current role is: LEADER
I20260808 14:31:31.701961 23309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17fe17e03d7a498897f8d082462f118a. Latest consensus state: current_term: 3 leader_uuid: "17fe17e03d7a498897f8d082462f118a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17fe17e03d7a498897f8d082462f118a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43375 } } }
I20260808 14:31:31.702118 23309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17fe17e03d7a498897f8d082462f118a [sys.catalog]: This master's current role is: LEADER
I20260808 14:31:31.702518 23314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:31:31.708317 23314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:31:31.708964 23314 catalog_manager.cc:1259] Loaded cluster ID: bb89ed74fe1d4d71973af55096723866
I20260808 14:31:31.709029 23314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 14:31:31.710232 23314 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6bb74b79ad google::LogMessage::Fail() at ??:0
@ 0x7f6bb74ba88a google::LogMessage::SendToLog() at ??:0
@ 0x7f6bb74b74de google::LogMessage::Flush() at ??:0
@ 0x7f6bb74b8a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6bc1df6fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f6bc1df7d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6bc1df4579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f6bc1e38640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6bb97093d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6bb9709c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f6bb970b3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6bb96fa512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6bbad3e6db start_thread at ??:0
@ 0x7f6bb5fb561f clone at ??:0
W20260808 14:31:31.808383 23305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:31.808660 23305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:31.808712 23305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:31.813087 23305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:31.813161 23305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:31.813225 23305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:31:31.818877 23305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:43435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:43577
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=32795
--tserver_master_addrs=127.7.125.126:43375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:31.820129 23305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:31.821210 23305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:31.832314 23346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:31.832377 23347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:31.832528 23349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:31.832648 23305 server_base.cc:1034] running on GCE node
I20260808 14:31:31.833210 23305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:31.833987 23305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:31.835167 23305 hybrid_clock.cc:648] HybridClock initialized: now 1786199491835151 us; error 31 us; skew 500 ppm
Aug 08 14:31:31 dist-test-slave-w206 krb5kdc[22088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199491, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:31.838203 23305 init.cc:377] Logged in from keytab as kudu/127.7.125.65@KRBTEST.COM (short username kudu)
I20260808 14:31:31.839308 23305 webserver.cc:460] Webserver started at http://127.7.125.65:32795/ using document root <none> and password file <none>
I20260808 14:31:31.839907 23305 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:31.839985 23305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:31.844198 23305 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:31:31.846521 23357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:31.847982 23305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:31:31.848085 23305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "2bf980017525452096a42bbd2ef96c08"
format_stamp: "Formatted at 2026-08-08 14:31:29 on dist-test-slave-w206"
server_key: "bdc5e36189de974eaa6bc4473d836da4"
server_key_iv: "aa9bfa2c378f6808fe240b0b56879196"
server_key_version: "encryptionkey@0"
I20260808 14:31:31.848703 23305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:31.859822 23305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:31.863088 23305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:31.863292 23305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:31.863900 23305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:31:31.865033 23365 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 14:31:31.870487 23305 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 14:31:31.870574 23305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260808 14:31:31.870680 23305 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 14:31:31.872419 23305 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 14:31:31.872499 23305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260808 14:31:31.872804 23365 tablet_bootstrap.cc:492] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08: Bootstrap starting.
I20260808 14:31:31.883929 23305 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:43577
I20260808 14:31:31.883984 23478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:43577 every 8 connection(s)
I20260808 14:31:31.884931 23305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:31:31.886523 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 23305
I20260808 14:31:31.887058 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:42295
--local_ip_for_outbound_sockets=127.7.125.66
--tserver_master_addrs=127.7.125.126:43375
--webserver_port=32937
--webserver_interface=127.7.125.66
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:43435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:31:31.899155 23365 log.cc:824] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08: Log is configured to *not* fsync() on all Append() calls
I20260808 14:31:31.908308 23365 tablet_bootstrap.cc:492] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:31:31.908756 23365 tablet_bootstrap.cc:492] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08: Bootstrap complete.
I20260808 14:31:31.910287 23365 ts_tablet_manager.cc:1397] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08: Time spent bootstrapping tablet: real 0.038s user 0.026s sys 0.004s
I20260808 14:31:31.913727 23365 raft_consensus.cc:348] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:31.914297 23365 raft_consensus.cc:729] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bf980017525452096a42bbd2ef96c08, State: Initialized, Role: FOLLOWER
I20260808 14:31:31.914865 23365 consensus_queue.cc:260] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:31.915952 23365 ts_tablet_manager.cc:1428] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08: Time spent starting tablet: real 0.006s user 0.004s sys 0.004s
W20260808 14:31:31.929713 23481 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:31:31.887102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.65:39987)
0808 14:31:31.887761 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:31.887828 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:31.888964 (+ 1136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:31.928747 (+ 39783us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":163,"threads_started":1}
W20260808 14:31:31.930392 23479 heartbeater.cc:644] Failed to heartbeat to 127.7.125.126:43375 (0 consecutive failures): Network error: Failed to ping master at 127.7.125.126:43375: Client connection negotiation failed: client connection to 127.7.125.126:43375: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 14:31:31.931186 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:31.930858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.65:33253)
0808 14:31:31.931046 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:31.931119 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 14:31:31.931871 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:31.931649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.65:35091)
0808 14:31:31.931751 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:31.931799 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 14:31:31.932049 23479 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 14:31:32.018958 23482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:32.019358 23482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:32.019479 23482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:32.023671 23482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:32.023743 23482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:32.023820 23482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:31:32.029320 23482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:43435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:42295
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=32937
--tserver_master_addrs=127.7.125.126:43375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:32.030356 23482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:32.031384 23482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:32.041419 23490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:32.041411 23489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:32.041811 23482 server_base.cc:1034] running on GCE node
W20260808 14:31:32.041391 23492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:32.042248 23482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:32.042913 23482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:32.044137 23482 hybrid_clock.cc:648] HybridClock initialized: now 1786199492044120 us; error 25 us; skew 500 ppm
Aug 08 14:31:32 dist-test-slave-w206 krb5kdc[22088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199492, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:32.047086 23482 init.cc:377] Logged in from keytab as kudu/127.7.125.66@KRBTEST.COM (short username kudu)
I20260808 14:31:32.047961 23482 webserver.cc:460] Webserver started at http://127.7.125.66:32937/ using document root <none> and password file <none>
I20260808 14:31:32.048491 23482 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:32.048553 23482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:32.052591 23482 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:31:32.054723 23499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:32.056042 23482 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:31:32.056175 23482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "9c2714e1b2cf4ada877e1c1a992c61a0"
format_stamp: "Formatted at 2026-08-08 14:31:29 on dist-test-slave-w206"
server_key: "941c2c7c1969bfbc408b56d3949c96da"
server_key_iv: "e2d3a92a2a2d5621794d8117060f591c"
server_key_version: "encryptionkey@0"
I20260808 14:31:32.056730 23482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:32.073719 23482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:32.076928 23482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:32.077142 23482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:32.077785 23482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:31:32.078665 23482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:31:32.078712 23482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:32.078783 23482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:31:32.078826 23482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:32.088559 23482 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:42295
I20260808 14:31:32.088582 23621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:42295 every 8 connection(s)
I20260808 14:31:32.089638 23482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:31:32.093438 23624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:32.091835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.66:56591)
0808 14:31:32.092309 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:32.092738 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":114,"threads_started":1}
W20260808 14:31:32.093870 23622 heartbeater.cc:644] Failed to heartbeat to 127.7.125.126:43375 (0 consecutive failures): Network error: Failed to ping master at 127.7.125.126:43375: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
I20260808 14:31:32.094462 23624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:32.094260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.66:49623)
0808 14:31:32.094365 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:32.094412 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 14:31:32.095069 23624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:32.094820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.66:44989)
0808 14:31:32.094922 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:32.094961 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 14:31:32.095242 23622 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 14:31:32.096441 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 23482
I20260808 14:31:32.096858 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:44891
--local_ip_for_outbound_sockets=127.7.125.67
--tserver_master_addrs=127.7.125.126:43375
--webserver_port=39557
--webserver_interface=127.7.125.67
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:43435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:31:32.211954 23625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:32.212244 23625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:32.212289 23625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:32.216596 23625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:32.216668 23625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:32.216732 23625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:31:32.222446 23625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:43435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:44891
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=39557
--tserver_master_addrs=127.7.125.126:43375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:32.223699 23625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:32.224772 23625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:32.236348 23633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:32.236378 23631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:32.236357 23630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:32.236856 23625 server_base.cc:1034] running on GCE node
I20260808 14:31:32.237375 23625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:32.238130 23625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:32.239333 23625 hybrid_clock.cc:648] HybridClock initialized: now 1786199492239312 us; error 39 us; skew 500 ppm
Aug 08 14:31:32 dist-test-slave-w206 krb5kdc[22088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199492, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:32.242451 23625 init.cc:377] Logged in from keytab as kudu/127.7.125.67@KRBTEST.COM (short username kudu)
I20260808 14:31:32.243430 23625 webserver.cc:460] Webserver started at http://127.7.125.67:39557/ using document root <none> and password file <none>
I20260808 14:31:32.244055 23625 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:32.244122 23625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:32.248031 23625 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:31:32.250080 23640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:32.251386 23625 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:31:32.251538 23625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "135b2386669b4430a003c3cf6dbb7728"
format_stamp: "Formatted at 2026-08-08 14:31:30 on dist-test-slave-w206"
server_key: "b21110fd0531d3fa3825ca7841d782dd"
server_key_iv: "1c8f7a1c7540a29ca6f1d58fade22d8e"
server_key_version: "encryptionkey@0"
I20260808 14:31:32.252131 23625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:32.276922 23625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:32.280087 23625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:32.280308 23625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:32.280939 23625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:31:32.281791 23625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:31:32.281862 23625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:32.281961 23625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:31:32.282012 23625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:32.291242 23625 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:44891
I20260808 14:31:32.291273 23759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:44891 every 8 connection(s)
I20260808 14:31:32.292309 23625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:31:32.294940 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 23625
I20260808 14:31:32.296205 23762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:32.294497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.67:33539)
0808 14:31:32.295287 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:32.295425 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":139,"threads_started":1}
W20260808 14:31:32.296667 23760 heartbeater.cc:644] Failed to heartbeat to 127.7.125.126:43375 (0 consecutive failures): Network error: Failed to ping master at 127.7.125.126:43375: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
I20260808 14:31:32.297251 23762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:32.297050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.67:39695)
0808 14:31:32.297152 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:32.297199 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 14:31:32.297845 23762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:32.297653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.67:39137)
0808 14:31:32.297756 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:32.297800 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 14:31:32.298024 23760 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 14:31:32.933079 23764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:32.932647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.65:59731)
0808 14:31:32.932895 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:32.932964 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":75,"threads_started":1}
I20260808 14:31:33.096302 23765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:33.095840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.66:60373)
0808 14:31:33.096137 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:33.096199 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":110,"threads_started":1}
I20260808 14:31:33.299130 23766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:33.298607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.67:39837)
0808 14:31:33.298874 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:33.298966 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":92,"threads_started":1}
I20260808 14:31:33.780908 23767 raft_consensus.cc:482] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:31:33.781100 23767 raft_consensus.cc:504] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:33.782107 23767 leader_election.cc:288] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 135b2386669b4430a003c3cf6dbb7728 (127.7.125.67:44891), 9c2714e1b2cf4ada877e1c1a992c61a0 (127.7.125.66:42295)
Aug 08 14:31:33 dist-test-slave-w206 krb5kdc[22088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199491, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.66@KRBTEST.COM
Aug 08 14:31:33 dist-test-slave-w206 krb5kdc[22088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199491, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.67@KRBTEST.COM
I20260808 14:31:33.796224 23770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:33.782753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:42295 (local address 127.7.125.65:48349)
0808 14:31:33.783064 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:33.783079 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:33.783179 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:33.784537 (+ 1358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:33.784543 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:33.785069 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:33.785569 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:33.785582 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:33.786873 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:33.786877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:33.787906 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:33.787916 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:33.788679 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:33.788683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:33.788868 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:33.789532 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:33.789547 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:33.791343 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:33.793669 (+ 2326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:33.793675 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:33.793688 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:33.794045 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:33.794412 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:33.794414 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:33.794416 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:33.794557 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:33.795256 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:33.795262 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:33.795494 (+ 232us) client_negotiation.cc:769] Sending connection context
0808 14:31:33.795676 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 14:31:33.796015 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":182,"spinlock_wait_cycles":39808,"thread_start_us":80,"threads_started":1}
I20260808 14:31:33.796248 23771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:33.782948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:48349 (local address 127.7.125.66:42295)
0808 14:31:33.783302 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:33.783312 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:33.783343 (+ 31us) server_negotiation.cc:407] Connection header received
0808 14:31:33.783955 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:33.783960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:33.784237 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:33.784542 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:33.785752 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:33.786720 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:33.788025 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:33.788524 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:33.788717 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:33.791592 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:33.791597 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:33.791609 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:33.791639 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:33.793522 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:33.794168 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:33.794181 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:33.794185 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:33.794299 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:33.794660 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:33.794663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:33.794665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:33.795130 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:33.795268 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:33.795797 (+ 529us) server_negotiation.cc:299] Negotiation successful
0808 14:31:33.796085 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":132,"threads_started":1}
I20260808 14:31:33.796277 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:33.782514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:57475 (local address 127.7.125.67:44891)
0808 14:31:33.782951 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:33.783003 (+ 52us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:33.783052 (+ 49us) server_negotiation.cc:407] Connection header received
0808 14:31:33.783799 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:33.783805 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:33.784145 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:33.784386 (+ 241us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:33.785754 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:33.786732 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:33.788063 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:33.788500 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:33.788676 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:33.791899 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:33.791904 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:33.791914 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:33.791942 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:33.793590 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:33.794189 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:33.794197 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:33.794202 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:33.794325 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:33.794662 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:33.794667 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:33.794669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:33.795131 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:33.795268 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:33.795796 (+ 528us) server_negotiation.cc:299] Negotiation successful
0808 14:31:33.796091 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":80,"threads_started":1}
I20260808 14:31:33.796265 23768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:33.782366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:44891 (local address 127.7.125.65:57475)
0808 14:31:33.782611 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:33.782626 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:33.782731 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:33.784536 (+ 1805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:33.784542 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:33.785075 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:33.785569 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:33.785583 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:33.786889 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:33.786892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:33.787912 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:33.787922 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:33.788649 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:33.788653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:33.788874 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:33.789584 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:33.789596 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:33.791719 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:33.793709 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:33.793713 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:33.793715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:33.794037 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:33.794429 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:33.794435 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:33.794439 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:33.794564 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:33.795255 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:33.795261 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:33.795494 (+ 233us) client_negotiation.cc:769] Sending connection context
0808 14:31:33.795677 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 14:31:33.796104 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":323,"spinlock_wait_cycles":21248,"thread_start_us":98,"threads_started":1}
W20260808 14:31:33.798923 23359 leader_election.cc:341] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2714e1b2cf4ada877e1c1a992c61a0 (127.7.125.66:42295): Not found: Tablet not found: f8be0d6da9be4a2fa126b5b52f7d6783
W20260808 14:31:33.799201 23359 leader_election.cc:341] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 135b2386669b4430a003c3cf6dbb7728 (127.7.125.67:44891): Not found: Tablet not found: f8be0d6da9be4a2fa126b5b52f7d6783
I20260808 14:31:33.799280 23359 leader_election.cc:302] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2bf980017525452096a42bbd2ef96c08; no voters: 135b2386669b4430a003c3cf6dbb7728, 9c2714e1b2cf4ada877e1c1a992c61a0
I20260808 14:31:33.799569 23767 raft_consensus.cc:2726] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 14:31:33.934195 23770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:33.933862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.65:41755)
0808 14:31:33.934022 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:33.934089 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 14:31:34.097682 23772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:34.097216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.66:43049)
0808 14:31:34.097502 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:34.097568 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":79,"threads_started":1}
I20260808 14:31:34.300246 23773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:34.299822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.67:41833)
0808 14:31:34.300096 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:34.300155 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":119,"threads_started":1}
I20260808 14:31:34.935541 23774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:34.935109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.65:57145)
0808 14:31:34.935366 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:34.935439 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":74,"threads_started":1}
I20260808 14:31:35.098798 23775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:35.098382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.66:48703)
0808 14:31:35.098655 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:35.098709 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":92,"threads_started":1}
I20260808 14:31:35.301437 23776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:35.300991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.67:48381)
0808 14:31:35.301274 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:35.301338 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":73,"threads_started":1}
I20260808 14:31:35.479645 23777 raft_consensus.cc:482] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:31:35.479789 23777 raft_consensus.cc:504] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:35.480216 23777 leader_election.cc:288] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 135b2386669b4430a003c3cf6dbb7728 (127.7.125.67:44891), 9c2714e1b2cf4ada877e1c1a992c61a0 (127.7.125.66:42295)
W20260808 14:31:35.481236 23359 leader_election.cc:341] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2714e1b2cf4ada877e1c1a992c61a0 (127.7.125.66:42295): Not found: Tablet not found: f8be0d6da9be4a2fa126b5b52f7d6783
W20260808 14:31:35.481392 23359 leader_election.cc:341] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 135b2386669b4430a003c3cf6dbb7728 (127.7.125.67:44891): Not found: Tablet not found: f8be0d6da9be4a2fa126b5b52f7d6783
I20260808 14:31:35.481458 23359 leader_election.cc:302] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2bf980017525452096a42bbd2ef96c08; no voters: 135b2386669b4430a003c3cf6dbb7728, 9c2714e1b2cf4ada877e1c1a992c61a0
I20260808 14:31:35.481633 23777 raft_consensus.cc:2726] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 14:31:35.936791 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:35.936360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.65:48475)
0808 14:31:35.936625 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:35.936680 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":105,"threads_started":1}
I20260808 14:31:36.100046 23779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:36.099640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.66:53531)
0808 14:31:36.099893 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:36.099957 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":83,"threads_started":1}
I20260808 14:31:36.302779 23780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:36.302332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.67:39437)
0808 14:31:36.302620 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:36.302683 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":111,"threads_started":1}
I20260808 14:31:36.938325 23781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:36.937849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.65:52801)
0808 14:31:36.938168 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:36.938228 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":75,"threads_started":1}
I20260808 14:31:37.101513 23782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:37.101057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.66:54607)
0808 14:31:37.101344 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:37.101419 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":102,"threads_started":1}
I20260808 14:31:37.304066 23783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:37.303638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.67:45341)
0808 14:31:37.303911 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:37.303970 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":76,"threads_started":1}
I20260808 14:31:37.723855 23785 raft_consensus.cc:482] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:31:37.723990 23785 raft_consensus.cc:504] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:37.724367 23785 leader_election.cc:288] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 135b2386669b4430a003c3cf6dbb7728 (127.7.125.67:44891), 9c2714e1b2cf4ada877e1c1a992c61a0 (127.7.125.66:42295)
W20260808 14:31:37.725199 23359 leader_election.cc:341] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 135b2386669b4430a003c3cf6dbb7728 (127.7.125.67:44891): Not found: Tablet not found: f8be0d6da9be4a2fa126b5b52f7d6783
W20260808 14:31:37.725364 23359 leader_election.cc:341] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2714e1b2cf4ada877e1c1a992c61a0 (127.7.125.66:42295): Not found: Tablet not found: f8be0d6da9be4a2fa126b5b52f7d6783
I20260808 14:31:37.725437 23359 leader_election.cc:302] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2bf980017525452096a42bbd2ef96c08; no voters: 135b2386669b4430a003c3cf6dbb7728, 9c2714e1b2cf4ada877e1c1a992c61a0
I20260808 14:31:37.725615 23785 raft_consensus.cc:2726] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 14:31:37.939639 23786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:37.939208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.65:41183)
0808 14:31:37.939489 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:37.939542 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":113,"threads_started":1}
I20260808 14:31:38.102830 23787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:38.102355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.66:57335)
0808 14:31:38.102664 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:38.102730 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":82,"threads_started":1}
I20260808 14:31:38.305320 23788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:38.304879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.67:44851)
0808 14:31:38.305168 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:38.305223 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":96,"threads_started":1}
I20260808 14:31:38.940914 23789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:38.940469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.65:37881)
0808 14:31:38.940738 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:38.940803 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":81,"threads_started":1}
I20260808 14:31:39.104192 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:39.103699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.66:51153)
0808 14:31:39.104027 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:39.104093 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":82,"threads_started":1}
I20260808 14:31:39.306663 23791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:39.306175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.67:43747)
0808 14:31:39.306506 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:39.306566 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":98,"threads_started":1}
I20260808 14:31:39.942234 23792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:39.941742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.65:42505)
0808 14:31:39.942067 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:39.942133 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":109,"threads_started":1}
I20260808 14:31:40.105522 23793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:40.105024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.66:47831)
0808 14:31:40.105351 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:40.105419 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":75,"threads_started":1}
I20260808 14:31:40.307840 23794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:40.307411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.67:43707)
0808 14:31:40.307677 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:40.307741 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":71,"threads_started":1}
I20260808 14:31:40.943617 23795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:40.943175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.65:45743)
0808 14:31:40.943455 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:40.943518 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":81,"threads_started":1}
I20260808 14:31:40.995184 23796 raft_consensus.cc:482] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:31:40.995352 23796 raft_consensus.cc:504] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135b2386669b4430a003c3cf6dbb7728" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 44891 } } peers { permanent_uuid: "9c2714e1b2cf4ada877e1c1a992c61a0" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 42295 } } peers { permanent_uuid: "2bf980017525452096a42bbd2ef96c08" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43577 } }
I20260808 14:31:40.995792 23796 leader_election.cc:288] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 135b2386669b4430a003c3cf6dbb7728 (127.7.125.67:44891), 9c2714e1b2cf4ada877e1c1a992c61a0 (127.7.125.66:42295)
W20260808 14:31:40.996683 23359 leader_election.cc:341] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 135b2386669b4430a003c3cf6dbb7728 (127.7.125.67:44891): Not found: Tablet not found: f8be0d6da9be4a2fa126b5b52f7d6783
W20260808 14:31:40.996855 23359 leader_election.cc:341] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2714e1b2cf4ada877e1c1a992c61a0 (127.7.125.66:42295): Not found: Tablet not found: f8be0d6da9be4a2fa126b5b52f7d6783
I20260808 14:31:40.996933 23359 leader_election.cc:302] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2bf980017525452096a42bbd2ef96c08; no voters: 135b2386669b4430a003c3cf6dbb7728, 9c2714e1b2cf4ada877e1c1a992c61a0
I20260808 14:31:40.997120 23796 raft_consensus.cc:2726] T f8be0d6da9be4a2fa126b5b52f7d6783 P 2bf980017525452096a42bbd2ef96c08 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 14:31:41.106784 23797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:41.106341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.66:40059)
0808 14:31:41.106621 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:41.106685 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":95,"threads_started":1}
I20260808 14:31:41.309057 23799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:41.308594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.67:59939)
0808 14:31:41.308876 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:41.308957 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":101,"threads_started":1}
I20260808 14:31:41.944963 23800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:41.944535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.65:34199)
0808 14:31:41.944788 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:41.944848 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":73,"threads_started":1}
I20260808 14:31:42.108177 23801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:42.107704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.66:34469)
0808 14:31:42.108015 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:42.108078 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":104,"threads_started":1}
I20260808 14:31:42.310459 23802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:42.310009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.67:54941)
0808 14:31:42.310289 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:42.310354 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":78,"threads_started":1}
I20260808 14:31:42.946192 23803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:42.945775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.65:33279)
0808 14:31:42.946035 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:42.946094 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":77,"threads_started":1}
I20260808 14:31:43.109519 23804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:43.109079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.66:35771)
0808 14:31:43.109364 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:43.109423 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":74,"threads_started":1}
I20260808 14:31:43.311764 23805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:43.311336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.67:44733)
0808 14:31:43.311600 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:43.311664 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":81,"threads_started":1}
I20260808 14:31:43.947479 23806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:43.947049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.65:45197)
0808 14:31:43.947313 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:43.947376 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":75,"threads_started":1}
I20260808 14:31:44.110844 23807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:44.110337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.66:37081)
0808 14:31:44.110689 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:44.110748 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":90,"threads_started":1}
I20260808 14:31:44.313146 23810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:44.312584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.67:37453)
0808 14:31:44.312976 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:44.313046 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":76,"threads_started":1}
I20260808 14:31:44.948701 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:44.948278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.65:57697)
0808 14:31:44.948542 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:44.948603 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":80,"threads_started":1}
I20260808 14:31:45.112169 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:45.111730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.66:59953)
0808 14:31:45.112007 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:45.112072 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":78,"threads_started":1}
I20260808 14:31:45.314582 23813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:45.314130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.67:39975)
0808 14:31:45.314412 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:45.314481 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":76,"threads_started":1}
I20260808 14:31:45.949994 23814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:45.949548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.65:44323)
0808 14:31:45.949823 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:45.949895 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":81,"threads_started":1}
I20260808 14:31:46.113462 23815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:46.112960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.66:58795)
0808 14:31:46.113287 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:46.113348 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":102,"threads_started":1}
I20260808 14:31:46.315907 23816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:46.315473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.67:40457)
0808 14:31:46.315739 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:46.315806 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":77,"threads_started":1}
I20260808 14:31:46.951524 23818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:46.951065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.65:35525)
0808 14:31:46.951314 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:46.951374 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":73,"threads_started":1}
I20260808 14:31:47.114737 23819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:47.114295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43375 (local address 127.7.125.66:40219)
0808 14:31:47.114574 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:47.114640 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:43375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":122,"threads_started":1}
I20260808 14:31:47.297719 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 23305
I20260808 14:31:47.305924 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 23482
I20260808 14:31:47.313644 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 23625
2026-08-08T14:31:47Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19400 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:31:47 dist-test-slave-w206 krb5kdc[23822](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:31:47 dist-test-slave-w206 krb5kdc[23822](info): set up 2 sockets
Aug 08 14:31:47 dist-test-slave-w206 krb5kdc[23822](info): commencing operation
krb5kdc: starting...
W20260808 14:31:48.392076 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:31:48 dist-test-slave-w206 krb5kdc[23822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:31:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:31:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:31:48.526782 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:35021
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:35021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:31:48.638682 23836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:48.639052 23836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:48.639173 23836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:48.643209 23836 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 14:31:48.643302 23836 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 14:31:48.643330 23836 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 14:31:48.643355 23836 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:31:48.643394 23836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:48.643436 23836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:48.643473 23836 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:31:48.643510 23836 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:31:48.650074 23836 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:35021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:35021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:48.651314 23836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:48.652356 23836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:48.661554 23843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:48.661649 23836 server_base.cc:1034] running on GCE node
W20260808 14:31:48.661583 23845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:48.661554 23841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:48.662356 23836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:48.663625 23836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:48.664819 23836 hybrid_clock.cc:648] HybridClock initialized: now 1786199508664801 us; error 35 us; skew 500 ppm
Aug 08 14:31:48 dist-test-slave-w206 krb5kdc[23822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199508, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:48.672562 23836 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:31:48.673609 23836 webserver.cc:460] Webserver started at http://127.7.125.126:41987/ using document root <none> and password file <none>
I20260808 14:31:48.674198 23836 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:48.674288 23836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:48.674533 23836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:31:48.676739 23836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "df1f026924554d6296c793aa96c6efc1"
format_stamp: "Formatted at 2026-08-08 14:31:48 on dist-test-slave-w206"
server_key: "4c99631163c88b253cbefcde2f5bc376"
server_key_iv: "1913ccbab000310daea50975dae2ff2a"
server_key_version: "encryptionkey@0"
I20260808 14:31:48.677307 23836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "df1f026924554d6296c793aa96c6efc1"
format_stamp: "Formatted at 2026-08-08 14:31:48 on dist-test-slave-w206"
server_key: "4c99631163c88b253cbefcde2f5bc376"
server_key_iv: "1913ccbab000310daea50975dae2ff2a"
server_key_version: "encryptionkey@0"
I20260808 14:31:48.681114 23836 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260808 14:31:48.683634 23853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:48.684922 23836 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:31:48.685066 23836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "df1f026924554d6296c793aa96c6efc1"
format_stamp: "Formatted at 2026-08-08 14:31:48 on dist-test-slave-w206"
server_key: "4c99631163c88b253cbefcde2f5bc376"
server_key_iv: "1913ccbab000310daea50975dae2ff2a"
server_key_version: "encryptionkey@0"
I20260808 14:31:48.685186 23836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:48.701287 23836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:48.704227 23836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:48.704450 23836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:48.711773 23836 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:35021
I20260808 14:31:48.712108 23907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:35021 every 8 connection(s)
I20260808 14:31:48.712790 23836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:31:48.714160 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 23836
I20260808 14:31:48.714257 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:31:48.714581 7669 external_mini_cluster.cc:1428] Setting key 66b3493b49e2a10f1694d6f40571e95c
I20260808 14:31:48.715992 23908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:31:48.722791 23908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1: Bootstrap starting.
Aug 08 14:31:48 dist-test-slave-w206 krb5kdc[23822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:31:48.725507 23908 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1: Neither blocks nor log segments found. Creating new log.
I20260808 14:31:48.726274 23908 log.cc:824] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1: Log is configured to *not* fsync() on all Append() calls
I20260808 14:31:48.728487 23908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1: No bootstrap required, opened a new log
I20260808 14:31:48.729348 23911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:48.716041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53162 (local address 127.7.125.126:35021)
0808 14:31:48.716510 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:48.716526 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:48.716562 (+ 36us) server_negotiation.cc:407] Connection header received
0808 14:31:48.717199 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:48.717214 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:48.717542 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:48.717820 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:48.718786 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:48.719839 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:48.721027 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:48.721497 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:48.721611 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:48.724551 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:48.724556 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:48.724568 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:48.724599 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:48.726334 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:48.726832 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:48.726837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:48.726842 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:48.726960 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:48.727351 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:48.727355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:48.727357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:48.727767 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:48.727991 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:48.728263 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 14:31:48.728611 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":149,"threads_started":1}
I20260808 14:31:48.732402 23908 raft_consensus.cc:348] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f026924554d6296c793aa96c6efc1" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35021 } }
I20260808 14:31:48.732734 23908 raft_consensus.cc:374] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:31:48.732820 23908 raft_consensus.cc:729] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df1f026924554d6296c793aa96c6efc1, State: Initialized, Role: FOLLOWER
I20260808 14:31:48.733230 23908 consensus_queue.cc:260] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f026924554d6296c793aa96c6efc1" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35021 } }
I20260808 14:31:48.733383 23908 raft_consensus.cc:388] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:31:48.733484 23908 raft_consensus.cc:482] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:31:48.733584 23908 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:31:48.734737 23908 raft_consensus.cc:504] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f026924554d6296c793aa96c6efc1" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35021 } }
I20260808 14:31:48.735143 23908 leader_election.cc:302] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df1f026924554d6296c793aa96c6efc1; no voters:
I20260808 14:31:48.735481 23908 leader_election.cc:288] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:31:48.735615 23913 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:31:48.735854 23913 raft_consensus.cc:686] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 1 LEADER]: Becoming Leader. State: Replica: df1f026924554d6296c793aa96c6efc1, State: Running, Role: LEADER
I20260808 14:31:48.736199 23913 consensus_queue.cc:237] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f026924554d6296c793aa96c6efc1" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35021 } }
I20260808 14:31:48.736747 23908 sys_catalog.cc:564] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:31:48.737756 23916 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df1f026924554d6296c793aa96c6efc1. Latest consensus state: current_term: 1 leader_uuid: "df1f026924554d6296c793aa96c6efc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f026924554d6296c793aa96c6efc1" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35021 } } }
I20260808 14:31:48.737890 23916 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [sys.catalog]: This master's current role is: LEADER
I20260808 14:31:48.738236 23914 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df1f026924554d6296c793aa96c6efc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f026924554d6296c793aa96c6efc1" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35021 } } }
I20260808 14:31:48.738317 23914 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [sys.catalog]: This master's current role is: LEADER
I20260808 14:31:48.738301 23923 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:31:48.741189 23923 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:31:48.746833 23923 catalog_manager.cc:1347] Generated new cluster ID: d3d1526fb6364165bb3538f7cb0b0049
I20260808 14:31:48.746905 23923 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:31:48.765746 23923 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:31:48.766834 23923 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:31:48.780718 23923 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1: Generated new TSK 0
I20260808 14:31:48.781600 23923 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.125.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
I20260808 14:31:48.819199 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:35021
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:31:48.935262 23935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:48.935560 23935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:48.935607 23935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:48.940057 23935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:48.940120 23935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:48.940179 23935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:31:48.945498 23935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:35021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:48.946630 23935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:48.947767 23935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:48.957711 23944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:48.957716 23940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:48.957736 23941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:48.958220 23935 server_base.cc:1034] running on GCE node
I20260808 14:31:48.958629 23935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:48.959367 23935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:48.960577 23935 hybrid_clock.cc:648] HybridClock initialized: now 1786199508960555 us; error 34 us; skew 500 ppm
Aug 08 14:31:48 dist-test-slave-w206 krb5kdc[23822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199508, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:48.963550 23935 init.cc:377] Logged in from keytab as kudu/127.7.125.65@KRBTEST.COM (short username kudu)
I20260808 14:31:48.964471 23935 webserver.cc:460] Webserver started at http://127.7.125.65:33109/ using document root <none> and password file <none>
I20260808 14:31:48.965044 23935 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:48.965133 23935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:48.965379 23935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:31:48.967607 23935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "e9894455cfa84afbbcf278234366d04f"
format_stamp: "Formatted at 2026-08-08 14:31:48 on dist-test-slave-w206"
server_key: "2c46c7f4d559287895aff88a7fc342ec"
server_key_iv: "f5990f533b62e3ba33492e42e8a0ecd3"
server_key_version: "encryptionkey@0"
I20260808 14:31:48.968189 23935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "e9894455cfa84afbbcf278234366d04f"
format_stamp: "Formatted at 2026-08-08 14:31:48 on dist-test-slave-w206"
server_key: "2c46c7f4d559287895aff88a7fc342ec"
server_key_iv: "f5990f533b62e3ba33492e42e8a0ecd3"
server_key_version: "encryptionkey@0"
I20260808 14:31:48.972038 23935 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 14:31:48.974542 23951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:48.976015 23935 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:31:48.976157 23935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "e9894455cfa84afbbcf278234366d04f"
format_stamp: "Formatted at 2026-08-08 14:31:48 on dist-test-slave-w206"
server_key: "2c46c7f4d559287895aff88a7fc342ec"
server_key_iv: "f5990f533b62e3ba33492e42e8a0ecd3"
server_key_version: "encryptionkey@0"
I20260808 14:31:48.976282 23935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:49.001430 23935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:49.004630 23935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:49.004858 23935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:49.005524 23935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:31:49.006397 23935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:31:49.006469 23935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:49.006543 23935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:31:49.006596 23935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:49.015941 23935 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:37689
I20260808 14:31:49.015965 24069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:37689 every 8 connection(s)
I20260808 14:31:49.016916 23935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:31:49.017283 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 23935
I20260808 14:31:49.017446 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:31:49.017788 7669 external_mini_cluster.cc:1428] Setting key 066ceddeff730252bf85d2a055e968c6
Aug 08 14:31:49 dist-test-slave-w206 krb5kdc[23822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199508, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.125.66@KRBTEST.COM; defaulting to no policy
I20260808 14:31:49.031713 23911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:49.018992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:43727 (local address 127.7.125.126:35021)
0808 14:31:49.019330 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:49.019335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:49.020396 (+ 1061us) server_negotiation.cc:407] Connection header received
0808 14:31:49.021352 (+ 956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:49.021355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:49.021409 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:49.021520 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:49.022680 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.023537 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.024816 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.025241 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.025389 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:49.027957 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:49.027961 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:49.027964 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:49.028001 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:49.029571 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:49.030174 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:49.030177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:49.030178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:49.030256 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:49.030560 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:49.030562 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:49.030563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:49.030744 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:49.030820 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:49.031431 (+ 611us) server_negotiation.cc:299] Negotiation successful
0808 14:31:49.031547 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.66@KRBTEST.COM" created.
I20260808 14:31:49.032881 24072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:49.019498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.65:43727)
0808 14:31:49.020243 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:49.020285 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:49.021144 (+ 859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:49.021694 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:49.021701 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:49.022064 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:49.022533 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:49.022546 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.023678 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.023681 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:49.024682 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:49.024692 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.025413 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.025416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:49.025565 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:49.026196 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:49.026214 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:49.027773 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:49.029682 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:49.029687 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:49.029700 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:49.030081 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:49.030340 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:49.030342 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:49.030343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:49.030480 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:49.030832 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:49.030836 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:49.031195 (+ 359us) client_negotiation.cc:769] Sending connection context
0808 14:31:49.031363 (+ 168us) client_negotiation.cc:240] Negotiation successful
0808 14:31:49.031635 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":141,"threads_started":1}
I20260808 14:31:49.034211 24070 heartbeater.cc:347] Connected to a master server at 127.7.125.126:35021
I20260808 14:31:49.034502 24070 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:49.035077 24070 heartbeater.cc:511] Master 127.7.125.126:35021 requested a full tablet report, sending...
I20260808 14:31:49.036631 23870 ts_manager.cc:194] Registered new tserver with Master: e9894455cfa84afbbcf278234366d04f (127.7.125.65:37689)
I20260808 14:31:49.038143 23870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:43727
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
I20260808 14:31:49.050014 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:35021
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:31:49.163463 24075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:49.163756 24075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:49.163837 24075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:49.167994 24075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:49.168071 24075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:49.168184 24075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:31:49.173805 24075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:35021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:49.174988 24075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:49.176028 24075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:31:49.185923 24081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:49.186024 24075 server_base.cc:1034] running on GCE node
W20260808 14:31:49.185971 24082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:49.185983 24084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:49.186688 24075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:49.187419 24075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:49.188606 24075 hybrid_clock.cc:648] HybridClock initialized: now 1786199509188594 us; error 33 us; skew 500 ppm
Aug 08 14:31:49 dist-test-slave-w206 krb5kdc[23822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199509, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:49.191430 24075 init.cc:377] Logged in from keytab as kudu/127.7.125.66@KRBTEST.COM (short username kudu)
I20260808 14:31:49.192425 24075 webserver.cc:460] Webserver started at http://127.7.125.66:33633/ using document root <none> and password file <none>
I20260808 14:31:49.193014 24075 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:49.193107 24075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:49.193348 24075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:31:49.195501 24075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "20c8a943104740ae80d2d8a4afd40732"
format_stamp: "Formatted at 2026-08-08 14:31:49 on dist-test-slave-w206"
server_key: "2a0ea25217641f33042ee4b10e4446e4"
server_key_iv: "896e27f5f284d47396fc98248cc5c282"
server_key_version: "encryptionkey@0"
I20260808 14:31:49.196089 24075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "20c8a943104740ae80d2d8a4afd40732"
format_stamp: "Formatted at 2026-08-08 14:31:49 on dist-test-slave-w206"
server_key: "2a0ea25217641f33042ee4b10e4446e4"
server_key_iv: "896e27f5f284d47396fc98248cc5c282"
server_key_version: "encryptionkey@0"
I20260808 14:31:49.199992 24075 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:31:49.202446 24091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:49.203785 24075 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:31:49.203933 24075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "20c8a943104740ae80d2d8a4afd40732"
format_stamp: "Formatted at 2026-08-08 14:31:49 on dist-test-slave-w206"
server_key: "2a0ea25217641f33042ee4b10e4446e4"
server_key_iv: "896e27f5f284d47396fc98248cc5c282"
server_key_version: "encryptionkey@0"
I20260808 14:31:49.204059 24075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:49.215926 24075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:49.218835 24075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:49.219103 24075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:49.219720 24075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:31:49.220583 24075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:31:49.220654 24075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:49.220726 24075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:31:49.220777 24075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:49.229550 24075 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:34299
I20260808 14:31:49.229576 24210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:34299 every 8 connection(s)
I20260808 14:31:49.230456 24075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:31:49.237367 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 24075
I20260808 14:31:49.237502 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:31:49.237855 7669 external_mini_cluster.cc:1428] Setting key 002488783d4e35192e04ce9b246e6cce
Aug 08 14:31:49 dist-test-slave-w206 krb5kdc[23822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199509, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:31:49.245002 23911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:49.232319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:46877 (local address 127.7.125.126:35021)
0808 14:31:49.232470 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:49.232474 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:49.233481 (+ 1007us) server_negotiation.cc:407] Connection header received
0808 14:31:49.234366 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:49.234369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:49.234438 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:49.234547 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:49.235763 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.236331 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.237768 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.238072 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.238196 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:49.241257 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:49.241261 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:49.241264 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:49.241291 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:49.242899 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:49.243467 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:49.243472 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:49.243474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:49.243557 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:49.243905 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:49.243908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:49.243910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:49.244092 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:49.244201 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:49.244694 (+ 493us) server_negotiation.cc:299] Negotiation successful
0808 14:31:49.244820 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:31:49.245783 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:49.232695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.66:46877)
0808 14:31:49.233336 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:49.233376 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:49.234164 (+ 788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:49.234715 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:49.234722 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:49.235153 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:49.235617 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:49.235632 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.236499 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.236502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:49.237619 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:49.237631 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.238387 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.238394 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:49.238632 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:49.239358 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:49.239379 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:49.241094 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:49.243050 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:49.243056 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:49.243071 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:49.243351 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:49.243649 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:49.243653 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:49.243656 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:49.243801 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:49.244199 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:49.244205 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:49.244459 (+ 254us) client_negotiation.cc:769] Sending connection context
0808 14:31:49.244642 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 14:31:49.244961 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":111,"threads_started":1}
I20260808 14:31:49.247076 24211 heartbeater.cc:347] Connected to a master server at 127.7.125.126:35021
I20260808 14:31:49.247316 24211 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:49.247795 24211 heartbeater.cc:511] Master 127.7.125.126:35021 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.125.67@KRBTEST.COM; defaulting to no policy
I20260808 14:31:49.248907 23870 ts_manager.cc:194] Registered new tserver with Master: 20c8a943104740ae80d2d8a4afd40732 (127.7.125.66:34299)
I20260808 14:31:49.249509 23870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:46877
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
I20260808 14:31:49.267863 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:35021
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:31:49.384492 24216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:31:49.384761 24216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:31:49.384809 24216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:31:49.388964 24216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:31:49.389037 24216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:31:49.389098 24216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:31:49.394764 24216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:35021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:31:49.395906 24216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:31:49.396932 24216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:31:49.407002 24216 server_base.cc:1034] running on GCE node
W20260808 14:31:49.406934 24222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:49.406905 24225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:31:49.407105 24223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:31:49.407621 24216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:31:49.408365 24216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:31:49.409560 24216 hybrid_clock.cc:648] HybridClock initialized: now 1786199509409545 us; error 30 us; skew 500 ppm
Aug 08 14:31:49 dist-test-slave-w206 krb5kdc[23822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199509, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:31:49.412420 24216 init.cc:377] Logged in from keytab as kudu/127.7.125.67@KRBTEST.COM (short username kudu)
I20260808 14:31:49.413309 24216 webserver.cc:460] Webserver started at http://127.7.125.67:34825/ using document root <none> and password file <none>
I20260808 14:31:49.413887 24216 fs_manager.cc:359] Metadata directory not provided
I20260808 14:31:49.413980 24216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:31:49.414217 24216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:31:49.416390 24216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "909465f4961c4c42b3a2f8876628cd13"
format_stamp: "Formatted at 2026-08-08 14:31:49 on dist-test-slave-w206"
server_key: "63477528bb8f9b7bdc99e2ff0740571a"
server_key_iv: "bcffdad8370fe02935ed5c6620853112"
server_key_version: "encryptionkey@0"
I20260808 14:31:49.416970 24216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "909465f4961c4c42b3a2f8876628cd13"
format_stamp: "Formatted at 2026-08-08 14:31:49 on dist-test-slave-w206"
server_key: "63477528bb8f9b7bdc99e2ff0740571a"
server_key_iv: "bcffdad8370fe02935ed5c6620853112"
server_key_version: "encryptionkey@0"
I20260808 14:31:49.420771 24216 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:31:49.423187 24232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:49.424752 24216 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:31:49.424896 24216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "909465f4961c4c42b3a2f8876628cd13"
format_stamp: "Formatted at 2026-08-08 14:31:49 on dist-test-slave-w206"
server_key: "63477528bb8f9b7bdc99e2ff0740571a"
server_key_iv: "bcffdad8370fe02935ed5c6620853112"
server_key_version: "encryptionkey@0"
I20260808 14:31:49.424998 24216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:31:49.447906 24216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:31:49.450759 24216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:31:49.450923 24216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:31:49.451551 24216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:31:49.452350 24216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:31:49.452399 24216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:49.452438 24216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:31:49.452454 24216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:31:49.461669 24216 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:34405
I20260808 14:31:49.461700 24352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:34405 every 8 connection(s)
I20260808 14:31:49.462550 24216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:31:49.465698 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 24216
I20260808 14:31:49.465811 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:31:49.466099 7669 external_mini_cluster.cc:1428] Setting key 496d5f0291a5b151f6b3c8d52d6a7d30
Aug 08 14:31:49 dist-test-slave-w206 krb5kdc[23822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199509, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:31:49.476547 23911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:49.464355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:60663 (local address 127.7.125.126:35021)
0808 14:31:49.464526 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:49.464531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:49.465351 (+ 820us) server_negotiation.cc:407] Connection header received
0808 14:31:49.466790 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:49.466793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:49.466851 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:49.466946 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:49.468650 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.469246 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.470345 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.470548 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.470620 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:49.473131 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:49.473135 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:49.473138 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:49.473171 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:49.474378 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:49.474916 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:49.474920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:49.474921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:49.475074 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:49.475442 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:49.475445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:49.475447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:49.475640 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:49.475750 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:49.476248 (+ 498us) server_negotiation.cc:299] Negotiation successful
0808 14:31:49.476374 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 14:31:49.477319 24355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:49.464755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.67:60663)
0808 14:31:49.465197 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:49.465241 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:49.466541 (+ 1300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:49.467230 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:49.467240 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:49.467794 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:31:49.468458 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:49.468501 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.469361 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.469364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:49.470240 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:49.470249 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.470677 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.470680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:49.470816 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:49.471382 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:31:49.471406 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:31:49.472963 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:31:49.474508 (+ 1545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:49.474515 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:49.474530 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:49.474808 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:49.475167 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:31:49.475171 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:31:49.475173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:31:49.475317 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:31:49.475749 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:31:49.475756 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:31:49.476004 (+ 248us) client_negotiation.cc:769] Sending connection context
0808 14:31:49.476191 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 14:31:49.476520 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":109,"threads_started":1}
I20260808 14:31:49.478408 24353 heartbeater.cc:347] Connected to a master server at 127.7.125.126:35021
I20260808 14:31:49.478653 24353 heartbeater.cc:464] Registering TS with master...
I20260808 14:31:49.479238 24353 heartbeater.cc:511] Master 127.7.125.126:35021 requested a full tablet report, sending...
I20260808 14:31:49.480355 23869 ts_manager.cc:194] Registered new tserver with Master: 909465f4961c4c42b3a2f8876628cd13 (127.7.125.67:34405)
I20260808 14:31:49.480945 23869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:60663
I20260808 14:31:49.492213 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:31:49.500941 23911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:49.494237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53168 (local address 127.7.125.126:35021)
0808 14:31:49.494355 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:49.494359 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:49.494563 (+ 204us) server_negotiation.cc:407] Connection header received
0808 14:31:49.494662 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:49.494664 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:49.494701 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:49.494761 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:31:49.495417 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.495918 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.496720 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.496928 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.497000 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:49.498246 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:31:49.498249 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:31:49.498251 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:31:49.498273 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:31:49.499508 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:49.499900 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:49.499903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:49.499904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:49.499987 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:31:49.500213 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:31:49.500216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:31:49.500217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:31:49.500424 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:31:49.500523 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:49.500713 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 14:31:49.500813 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 14:31:49.503816 23869 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:31:49.506795 23869 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:31:49.521235 24364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:49.515101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:34405 (local address 127.0.0.1:56646)
0808 14:31:49.515605 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:49.515654 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:49.515786 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:49.516661 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:49.516667 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:49.516690 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:31:49.516849 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:49.516857 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.518596 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.518600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:49.519824 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:49.519833 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.520677 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.520680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:49.520786 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:49.520822 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 14:31:49.520945 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 14:31:49.521066 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":45,"thread_start_us":62,"threads_started":1}
I20260808 14:31:49.521400 24369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:49.515967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53632 (local address 127.7.125.66:34299)
0808 14:31:49.516920 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:49.516934 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:49.516957 (+ 23us) server_negotiation.cc:407] Connection header received
0808 14:31:49.517030 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:49.517035 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:49.517200 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:49.517344 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:31:49.517559 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.518624 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.520005 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.520758 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.520911 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:49.521042 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:49.521142 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 14:31:49.521234 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":847,"thread_start_us":117,"threads_started":1}
I20260808 14:31:49.521569 24366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:49.515270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56646 (local address 127.7.125.67:34405)
0808 14:31:49.516127 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:49.516139 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:49.516165 (+ 26us) server_negotiation.cc:407] Connection header received
0808 14:31:49.516250 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:49.516258 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:49.516481 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:49.517015 (+ 534us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:31:49.517167 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.518461 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.519972 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.520539 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.520665 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:49.520816 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:49.521313 (+ 497us) server_negotiation.cc:299] Negotiation successful
0808 14:31:49.521418 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":753,"thread_start_us":156,"threads_started":1}
I20260808 14:31:49.521381 24365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:49.515743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:34299 (local address 127.0.0.1:53632)
0808 14:31:49.516575 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:49.516590 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:49.516664 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:49.517337 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:49.517340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:49.517359 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:31:49.517465 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:49.517471 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.518750 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.518754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:49.519863 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:49.519872 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.520903 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.520910 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:49.521016 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:49.521037 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 14:31:49.521112 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 14:31:49.521176 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"mutex_wait_us":32,"thread_start_us":94,"threads_started":1}
I20260808 14:31:49.522079 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:49.515297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:37689 (local address 127.0.0.1:37286)
0808 14:31:49.516227 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:49.516241 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:49.516339 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:49.516941 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:49.516944 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:49.516961 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:31:49.517065 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:49.517070 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.518176 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.518181 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:49.519603 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:49.519614 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.520889 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.520892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:49.520988 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:49.521010 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 14:31:49.521093 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 14:31:49.521173 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":871,"mutex_wait_us":39,"thread_start_us":122,"threads_started":1}
I20260808 14:31:49.522220 24368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:49.515862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37286 (local address 127.7.125.65:37689)
0808 14:31:49.516176 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:49.516191 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:49.516338 (+ 147us) server_negotiation.cc:407] Connection header received
0808 14:31:49.516451 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:49.516456 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:49.516633 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:49.516787 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:31:49.517176 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.518048 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.519734 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.520577 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.520684 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:49.520826 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:49.521798 (+ 972us) server_negotiation.cc:299] Negotiation successful
0808 14:31:49.521937 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":103,"threads_started":1}
I20260808 14:31:49.524717 24284 tablet_service.cc:1467] Processing CreateTablet for tablet cb89ef98ea4c4c34b742e64c26035499 (DEFAULT_TABLE table=test-table [id=57c469d7ccd944279d32729a3d4d2cec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:31:49.524950 24142 tablet_service.cc:1467] Processing CreateTablet for tablet cb89ef98ea4c4c34b742e64c26035499 (DEFAULT_TABLE table=test-table [id=57c469d7ccd944279d32729a3d4d2cec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:31:49.524948 24003 tablet_service.cc:1467] Processing CreateTablet for tablet cb89ef98ea4c4c34b742e64c26035499 (DEFAULT_TABLE table=test-table [id=57c469d7ccd944279d32729a3d4d2cec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:31:49.525745 24284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb89ef98ea4c4c34b742e64c26035499. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:31:49.525975 24142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb89ef98ea4c4c34b742e64c26035499. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:31:49.525975 24003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb89ef98ea4c4c34b742e64c26035499. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:31:49.532677 24370 tablet_bootstrap.cc:492] T cb89ef98ea4c4c34b742e64c26035499 P 909465f4961c4c42b3a2f8876628cd13: Bootstrap starting.
I20260808 14:31:49.533538 24372 tablet_bootstrap.cc:492] T cb89ef98ea4c4c34b742e64c26035499 P 20c8a943104740ae80d2d8a4afd40732: Bootstrap starting.
I20260808 14:31:49.535218 24370 tablet_bootstrap.cc:654] T cb89ef98ea4c4c34b742e64c26035499 P 909465f4961c4c42b3a2f8876628cd13: Neither blocks nor log segments found. Creating new log.
I20260808 14:31:49.535676 24372 tablet_bootstrap.cc:654] T cb89ef98ea4c4c34b742e64c26035499 P 20c8a943104740ae80d2d8a4afd40732: Neither blocks nor log segments found. Creating new log.
I20260808 14:31:49.535991 24370 log.cc:824] T cb89ef98ea4c4c34b742e64c26035499 P 909465f4961c4c42b3a2f8876628cd13: Log is configured to *not* fsync() on all Append() calls
I20260808 14:31:49.536458 24372 log.cc:824] T cb89ef98ea4c4c34b742e64c26035499 P 20c8a943104740ae80d2d8a4afd40732: Log is configured to *not* fsync() on all Append() calls
I20260808 14:31:49.537252 24371 tablet_bootstrap.cc:492] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f: Bootstrap starting.
I20260808 14:31:49.538288 24370 tablet_bootstrap.cc:492] T cb89ef98ea4c4c34b742e64c26035499 P 909465f4961c4c42b3a2f8876628cd13: No bootstrap required, opened a new log
I20260808 14:31:49.538519 24370 ts_tablet_manager.cc:1397] T cb89ef98ea4c4c34b742e64c26035499 P 909465f4961c4c42b3a2f8876628cd13: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 14:31:49.539412 24371 tablet_bootstrap.cc:654] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f: Neither blocks nor log segments found. Creating new log.
I20260808 14:31:49.540099 24371 log.cc:824] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f: Log is configured to *not* fsync() on all Append() calls
I20260808 14:31:49.541638 24372 tablet_bootstrap.cc:492] T cb89ef98ea4c4c34b742e64c26035499 P 20c8a943104740ae80d2d8a4afd40732: No bootstrap required, opened a new log
I20260808 14:31:49.541839 24372 ts_tablet_manager.cc:1397] T cb89ef98ea4c4c34b742e64c26035499 P 20c8a943104740ae80d2d8a4afd40732: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260808 14:31:49.541953 24371 tablet_bootstrap.cc:492] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f: No bootstrap required, opened a new log
I20260808 14:31:49.542131 24371 ts_tablet_manager.cc:1397] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 14:31:49.541923 24370 raft_consensus.cc:348] T cb89ef98ea4c4c34b742e64c26035499 P 909465f4961c4c42b3a2f8876628cd13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } } peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } }
I20260808 14:31:49.542217 24370 raft_consensus.cc:374] T cb89ef98ea4c4c34b742e64c26035499 P 909465f4961c4c42b3a2f8876628cd13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:31:49.542289 24370 raft_consensus.cc:729] T cb89ef98ea4c4c34b742e64c26035499 P 909465f4961c4c42b3a2f8876628cd13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 909465f4961c4c42b3a2f8876628cd13, State: Initialized, Role: FOLLOWER
I20260808 14:31:49.542734 24370 consensus_queue.cc:260] T cb89ef98ea4c4c34b742e64c26035499 P 909465f4961c4c42b3a2f8876628cd13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } } peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } }
I20260808 14:31:49.543429 24353 heartbeater.cc:503] Master 127.7.125.126:35021 was elected leader, sending a full tablet report...
I20260808 14:31:49.543692 24370 ts_tablet_manager.cc:1428] T cb89ef98ea4c4c34b742e64c26035499 P 909465f4961c4c42b3a2f8876628cd13: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260808 14:31:49.546439 24371 raft_consensus.cc:348] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } } peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } }
I20260808 14:31:49.546434 24372 raft_consensus.cc:348] T cb89ef98ea4c4c34b742e64c26035499 P 20c8a943104740ae80d2d8a4afd40732 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } } peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } }
I20260808 14:31:49.546705 24372 raft_consensus.cc:374] T cb89ef98ea4c4c34b742e64c26035499 P 20c8a943104740ae80d2d8a4afd40732 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:31:49.546705 24371 raft_consensus.cc:374] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:31:49.546766 24371 raft_consensus.cc:729] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9894455cfa84afbbcf278234366d04f, State: Initialized, Role: FOLLOWER
I20260808 14:31:49.546803 24372 raft_consensus.cc:729] T cb89ef98ea4c4c34b742e64c26035499 P 20c8a943104740ae80d2d8a4afd40732 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c8a943104740ae80d2d8a4afd40732, State: Initialized, Role: FOLLOWER
I20260808 14:31:49.547250 24371 consensus_queue.cc:260] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } } peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } }
I20260808 14:31:49.547250 24372 consensus_queue.cc:260] T cb89ef98ea4c4c34b742e64c26035499 P 20c8a943104740ae80d2d8a4afd40732 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } } peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } }
I20260808 14:31:49.548146 24070 heartbeater.cc:503] Master 127.7.125.126:35021 was elected leader, sending a full tablet report...
I20260808 14:31:49.548171 24211 heartbeater.cc:503] Master 127.7.125.126:35021 was elected leader, sending a full tablet report...
I20260808 14:31:49.548250 24371 ts_tablet_manager.cc:1428] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260808 14:31:49.548250 24372 ts_tablet_manager.cc:1428] T cb89ef98ea4c4c34b742e64c26035499 P 20c8a943104740ae80d2d8a4afd40732: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260808 14:31:49.713557 24354 tablet.cc:2369] T cb89ef98ea4c4c34b742e64c26035499 P 909465f4961c4c42b3a2f8876628cd13: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:31:49.731638 24212 tablet.cc:2369] T cb89ef98ea4c4c34b742e64c26035499 P 20c8a943104740ae80d2d8a4afd40732: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:31:49.768597 24071 tablet.cc:2369] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:31:49.778126 24378 raft_consensus.cc:482] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:31:49.778329 24378 raft_consensus.cc:504] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } } peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } }
I20260808 14:31:49.779774 24378 leader_election.cc:288] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 909465f4961c4c42b3a2f8876628cd13 (127.7.125.67:34405), 20c8a943104740ae80d2d8a4afd40732 (127.7.125.66:34299)
I20260808 14:31:49.784037 24366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:49.780239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:47201 (local address 127.7.125.67:34405)
0808 14:31:49.780474 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:49.780483 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:49.780788 (+ 305us) server_negotiation.cc:407] Connection header received
0808 14:31:49.780942 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:49.780945 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:49.781002 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:49.781087 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:31:49.781388 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.781910 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.783069 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.783460 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.783596 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:49.783652 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:49.783841 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 14:31:49.783937 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260808 14:31:49.784077 24381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:49.780141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:34405 (local address 127.7.125.65:47201)
0808 14:31:49.780649 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:49.780671 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:49.780812 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:49.781109 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:49.781111 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:49.781126 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:31:49.781273 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:49.781279 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.782023 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.782027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:49.782900 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:49.782911 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.783597 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.783601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:49.783695 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:49.783718 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 14:31:49.783811 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 14:31:49.783890 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":41,"thread_start_us":158,"threads_started":1}
I20260808 14:31:49.784220 24369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:49.780620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:44425 (local address 127.7.125.66:34299)
0808 14:31:49.780766 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:49.780772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:49.780942 (+ 170us) server_negotiation.cc:407] Connection header received
0808 14:31:49.781079 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:49.781083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:49.781136 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:49.781224 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:31:49.781492 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.782069 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.783283 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.783675 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.783805 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:49.783859 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:49.784042 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 14:31:49.784110 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 14:31:49.784236 24380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:49.780327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:34299 (local address 127.7.125.65:44425)
0808 14:31:49.780806 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:49.780822 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:31:49.780955 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:31:49.781236 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:31:49.781239 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:31:49.781255 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:31:49.781385 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:49.781391 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.782181 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.782186 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:49.783167 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:31:49.783176 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.783798 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.783802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:31:49.783922 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:49.783940 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 14:31:49.784027 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 14:31:49.784094 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":142,"threads_started":1}
I20260808 14:31:49.784844 24163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb89ef98ea4c4c34b742e64c26035499" candidate_uuid: "e9894455cfa84afbbcf278234366d04f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c8a943104740ae80d2d8a4afd40732" is_pre_election: true
I20260808 14:31:49.784845 24306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb89ef98ea4c4c34b742e64c26035499" candidate_uuid: "e9894455cfa84afbbcf278234366d04f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "909465f4961c4c42b3a2f8876628cd13" is_pre_election: true
I20260808 14:31:49.785166 24163 raft_consensus.cc:2454] T cb89ef98ea4c4c34b742e64c26035499 P 20c8a943104740ae80d2d8a4afd40732 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9894455cfa84afbbcf278234366d04f in term 0.
I20260808 14:31:49.785167 24306 raft_consensus.cc:2454] T cb89ef98ea4c4c34b742e64c26035499 P 909465f4961c4c42b3a2f8876628cd13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9894455cfa84afbbcf278234366d04f in term 0.
I20260808 14:31:49.785681 23956 leader_election.cc:302] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 20c8a943104740ae80d2d8a4afd40732, 909465f4961c4c42b3a2f8876628cd13, e9894455cfa84afbbcf278234366d04f; no voters:
I20260808 14:31:49.785988 24378 raft_consensus.cc:2781] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:31:49.786064 24378 raft_consensus.cc:482] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:31:49.786093 24378 raft_consensus.cc:3037] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:31:49.787209 24378 raft_consensus.cc:504] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } } peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } }
I20260808 14:31:49.787587 24378 leader_election.cc:288] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 1 election: Requested vote from peers 909465f4961c4c42b3a2f8876628cd13 (127.7.125.67:34405), 20c8a943104740ae80d2d8a4afd40732 (127.7.125.66:34299)
I20260808 14:31:49.787926 24163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb89ef98ea4c4c34b742e64c26035499" candidate_uuid: "e9894455cfa84afbbcf278234366d04f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c8a943104740ae80d2d8a4afd40732"
I20260808 14:31:49.787951 24306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb89ef98ea4c4c34b742e64c26035499" candidate_uuid: "e9894455cfa84afbbcf278234366d04f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "909465f4961c4c42b3a2f8876628cd13"
I20260808 14:31:49.788066 24163 raft_consensus.cc:3037] T cb89ef98ea4c4c34b742e64c26035499 P 20c8a943104740ae80d2d8a4afd40732 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:31:49.788061 24306 raft_consensus.cc:3037] T cb89ef98ea4c4c34b742e64c26035499 P 909465f4961c4c42b3a2f8876628cd13 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:31:49.789373 24163 raft_consensus.cc:2454] T cb89ef98ea4c4c34b742e64c26035499 P 20c8a943104740ae80d2d8a4afd40732 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9894455cfa84afbbcf278234366d04f in term 1.
I20260808 14:31:49.789373 24306 raft_consensus.cc:2454] T cb89ef98ea4c4c34b742e64c26035499 P 909465f4961c4c42b3a2f8876628cd13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9894455cfa84afbbcf278234366d04f in term 1.
I20260808 14:31:49.789736 23953 leader_election.cc:302] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c8a943104740ae80d2d8a4afd40732, e9894455cfa84afbbcf278234366d04f; no voters:
I20260808 14:31:49.789919 24378 raft_consensus.cc:2781] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:31:49.790177 24378 raft_consensus.cc:686] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [term 1 LEADER]: Becoming Leader. State: Replica: e9894455cfa84afbbcf278234366d04f, State: Running, Role: LEADER
I20260808 14:31:49.790527 24378 consensus_queue.cc:237] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } } peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } }
I20260808 14:31:49.793802 23869 catalog_manager.cc:5495] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f reported cstate change: term changed from 0 to 1, leader changed from <none> to e9894455cfa84afbbcf278234366d04f (127.7.125.65). New cstate: current_term: 1 leader_uuid: "e9894455cfa84afbbcf278234366d04f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } health_report { overall_health: UNKNOWN } } }
I20260808 14:31:49.813686 24368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:49.810010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37290 (local address 127.7.125.65:37689)
0808 14:31:49.810159 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 14:31:49.810164 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:31:49.810180 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:31:49.810298 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:31:49.810300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:31:49.810359 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:31:49.810479 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:31:49.810733 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.811276 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.812165 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:31:49.812386 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:31:49.812501 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:31:49.812821 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:31:49.813252 (+ 431us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:31:49.813389 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 14:31:49.813461 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 14:31:49.813523 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 14:31:49.819830 24163 raft_consensus.cc:1261] T cb89ef98ea4c4c34b742e64c26035499 P 20c8a943104740ae80d2d8a4afd40732 [term 1 FOLLOWER]: Refusing update from remote peer e9894455cfa84afbbcf278234366d04f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:31:49.819830 24306 raft_consensus.cc:1261] T cb89ef98ea4c4c34b742e64c26035499 P 909465f4961c4c42b3a2f8876628cd13 [term 1 FOLLOWER]: Refusing update from remote peer e9894455cfa84afbbcf278234366d04f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:31:49.820647 24378 consensus_queue.cc:1035] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [LEADER]: Connected to new peer: Peer: permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:31:49.820891 24382 consensus_queue.cc:1035] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:31:49.828673 24387 mvcc.cc:205] Tried to move back new op lower bound from 7316273192213786624 to 7316273192104882176. Current Snapshot: MvccSnapshot[applied={T|T < 7316273192213786624}]
I20260808 14:31:49.839349 23870 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:31:49.839536 23870 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:31:49.841950 23870 catalog_manager.cc:5782] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1: Sending DeleteTablet for 3 replicas of tablet cb89ef98ea4c4c34b742e64c26035499
I20260808 14:31:49.842806 24284 tablet_service.cc:1514] Processing DeleteTablet for tablet cb89ef98ea4c4c34b742e64c26035499 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:31:49 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:56646
I20260808 14:31:49.843168 24142 tablet_service.cc:1514] Processing DeleteTablet for tablet cb89ef98ea4c4c34b742e64c26035499 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:31:49 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:53632
I20260808 14:31:49.843400 24003 tablet_service.cc:1514] Processing DeleteTablet for tablet cb89ef98ea4c4c34b742e64c26035499 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:31:49 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:37286
I20260808 14:31:49.843500 24395 tablet_replica.cc:330] T cb89ef98ea4c4c34b742e64c26035499 P 909465f4961c4c42b3a2f8876628cd13: stopping tablet replica
I20260808 14:31:49.843724 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 23935
I20260808 14:31:49.845690 24395 raft_consensus.cc:2229] T cb89ef98ea4c4c34b742e64c26035499 P 909465f4961c4c42b3a2f8876628cd13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:31:49.846060 24395 raft_consensus.cc:2258] T cb89ef98ea4c4c34b742e64c26035499 P 909465f4961c4c42b3a2f8876628cd13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:31:49.848061 24396 tablet_replica.cc:330] T cb89ef98ea4c4c34b742e64c26035499 P 20c8a943104740ae80d2d8a4afd40732: stopping tablet replica
I20260808 14:31:49.848088 24395 ts_tablet_manager.cc:1905] T cb89ef98ea4c4c34b742e64c26035499 P 909465f4961c4c42b3a2f8876628cd13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:31:49.848455 24396 raft_consensus.cc:2229] T cb89ef98ea4c4c34b742e64c26035499 P 20c8a943104740ae80d2d8a4afd40732 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:31:49.848790 24396 raft_consensus.cc:2258] T cb89ef98ea4c4c34b742e64c26035499 P 20c8a943104740ae80d2d8a4afd40732 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:31:49.850095 24396 ts_tablet_manager.cc:1905] T cb89ef98ea4c4c34b742e64c26035499 P 20c8a943104740ae80d2d8a4afd40732: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:31:49.851859 24395 ts_tablet_manager.cc:1918] T cb89ef98ea4c4c34b742e64c26035499 P 909465f4961c4c42b3a2f8876628cd13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:31:49.852008 24395 log.cc:1196] T cb89ef98ea4c4c34b742e64c26035499 P 909465f4961c4c42b3a2f8876628cd13: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/cb89ef98ea4c4c34b742e64c26035499
I20260808 14:31:49.852380 24395 ts_tablet_manager.cc:1939] T cb89ef98ea4c4c34b742e64c26035499 P 909465f4961c4c42b3a2f8876628cd13: Deleting consensus metadata
I20260808 14:31:49.852383 24396 ts_tablet_manager.cc:1918] T cb89ef98ea4c4c34b742e64c26035499 P 20c8a943104740ae80d2d8a4afd40732: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:31:49.852504 24396 log.cc:1196] T cb89ef98ea4c4c34b742e64c26035499 P 20c8a943104740ae80d2d8a4afd40732: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/cb89ef98ea4c4c34b742e64c26035499
I20260808 14:31:49.852775 24396 ts_tablet_manager.cc:1939] T cb89ef98ea4c4c34b742e64c26035499 P 20c8a943104740ae80d2d8a4afd40732: Deleting consensus metadata
I20260808 14:31:49.853866 23857 catalog_manager.cc:4841] TS 909465f4961c4c42b3a2f8876628cd13 (127.7.125.67:34405): tablet cb89ef98ea4c4c34b742e64c26035499 (table test-table [id=57c469d7ccd944279d32729a3d4d2cec]) successfully deleted
I20260808 14:31:49.854017 23855 catalog_manager.cc:4841] TS 20c8a943104740ae80d2d8a4afd40732 (127.7.125.66:34299): tablet cb89ef98ea4c4c34b742e64c26035499 (table test-table [id=57c469d7ccd944279d32729a3d4d2cec]) successfully deleted
W20260808 14:31:49.854270 23856 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.125.65:37689 (error 108)
I20260808 14:31:49.855235 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 24075
I20260808 14:31:49.855437 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:31:49.855084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:37689 (local address 127.0.0.1:37302)
0808 14:31:49.855235 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 14:31:49.855323 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.65:37689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260808 14:31:49.857638 23856 catalog_manager.cc:4584] TS e9894455cfa84afbbcf278234366d04f (127.7.125.65:37689): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cb89ef98ea4c4c34b742e64c26035499: Network error: Client connection negotiation failed: client connection to 127.7.125.65:37689: connect: Connection refused (error 111)
I20260808 14:31:49.862665 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 24216
I20260808 14:31:49.869772 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 23836
I20260808 14:32:09.878463 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:35021
--webserver_interface=127.7.125.126
--webserver_port=41987
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:35021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:32:09.993444 24400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:09.993727 24400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:09.993803 24400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:09.998440 24400 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 14:32:09.998497 24400 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 14:32:09.998513 24400 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 14:32:09.998528 24400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:32:09.998548 24400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:09.998589 24400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:09.998623 24400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:32:09.998651 24400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:32:10.004132 24400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:35021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:35021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=41987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:10.005334 24400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:10.006429 24400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:10.021113 24406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:10.021229 24407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:10.021211 24400 server_base.cc:1034] running on GCE node
W20260808 14:32:10.021426 24409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:10.022107 24400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:10.023557 24400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:10.024765 24400 hybrid_clock.cc:648] HybridClock initialized: now 1786199530024728 us; error 57 us; skew 500 ppm
Aug 08 14:32:10 dist-test-slave-w206 krb5kdc[23822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199530, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:32:10.032639 24400 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:32:10.033747 24400 webserver.cc:460] Webserver started at http://127.7.125.126:41987/ using document root <none> and password file <none>
I20260808 14:32:10.034440 24400 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:10.034574 24400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:10.039151 24400 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:32:10.041661 24417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:10.043118 24400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:32:10.043279 24400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "df1f026924554d6296c793aa96c6efc1"
format_stamp: "Formatted at 2026-08-08 14:31:48 on dist-test-slave-w206"
server_key: "4c99631163c88b253cbefcde2f5bc376"
server_key_iv: "1913ccbab000310daea50975dae2ff2a"
server_key_version: "encryptionkey@0"
I20260808 14:32:10.043944 24400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:10.056529 24400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:10.060148 24400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:10.060376 24400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:10.067786 24400 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:35021
I20260808 14:32:10.068106 24475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:35021 every 8 connection(s)
I20260808 14:32:10.068979 24400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:32:10.073625 24476 sys_catalog.cc:263] Verifying existing consensus state
I20260808 14:32:10.075448 24476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1: Bootstrap starting.
I20260808 14:32:10.077018 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 24400
I20260808 14:32:10.077528 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:37689
--local_ip_for_outbound_sockets=127.7.125.65
--tserver_master_addrs=127.7.125.126:35021
--webserver_port=33109
--webserver_interface=127.7.125.65
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:32:10.098933 24476 log.cc:824] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:10.110644 24476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:32:10.111199 24476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1: Bootstrap complete.
I20260808 14:32:10.117069 24476 raft_consensus.cc:348] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f026924554d6296c793aa96c6efc1" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35021 } }
I20260808 14:32:10.117627 24476 raft_consensus.cc:729] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: df1f026924554d6296c793aa96c6efc1, State: Initialized, Role: FOLLOWER
I20260808 14:32:10.118202 24476 consensus_queue.cc:260] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f026924554d6296c793aa96c6efc1" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35021 } }
I20260808 14:32:10.118350 24476 raft_consensus.cc:388] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:32:10.118438 24476 raft_consensus.cc:482] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:32:10.118535 24476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 1 FOLLOWER]: Advancing to term 2
I20260808 14:32:10.121189 24476 raft_consensus.cc:504] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f026924554d6296c793aa96c6efc1" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35021 } }
I20260808 14:32:10.121587 24476 leader_election.cc:302] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df1f026924554d6296c793aa96c6efc1; no voters:
I20260808 14:32:10.121951 24476 leader_election.cc:288] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 14:32:10.122098 24481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 14:32:10.122399 24481 raft_consensus.cc:686] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 2 LEADER]: Becoming Leader. State: Replica: df1f026924554d6296c793aa96c6efc1, State: Running, Role: LEADER
I20260808 14:32:10.122721 24481 consensus_queue.cc:237] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f026924554d6296c793aa96c6efc1" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35021 } }
I20260808 14:32:10.123445 24476 sys_catalog.cc:564] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:32:10.124402 24483 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df1f026924554d6296c793aa96c6efc1. Latest consensus state: current_term: 2 leader_uuid: "df1f026924554d6296c793aa96c6efc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f026924554d6296c793aa96c6efc1" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35021 } } }
I20260808 14:32:10.124429 24482 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "df1f026924554d6296c793aa96c6efc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f026924554d6296c793aa96c6efc1" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35021 } } }
I20260808 14:32:10.124568 24482 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [sys.catalog]: This master's current role is: LEADER
I20260808 14:32:10.124568 24483 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [sys.catalog]: This master's current role is: LEADER
I20260808 14:32:10.128926 24497 catalog_manager.cc:1259] Loaded cluster ID: d3d1526fb6364165bb3538f7cb0b0049
I20260808 14:32:10.128994 24497 catalog_manager.cc:1552] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1: loading cluster ID for follower catalog manager: success
I20260808 14:32:10.131572 24497 catalog_manager.cc:1574] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1: acquiring CA information for follower catalog manager: success
I20260808 14:32:10.133160 24497 catalog_manager.cc:1602] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 14:32:10.133308 24490 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:32:10.135814 24490 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:32:10.136392 24490 catalog_manager.cc:1259] Loaded cluster ID: d3d1526fb6364165bb3538f7cb0b0049
I20260808 14:32:10.136462 24490 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:32:10.137157 24490 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:32:10.137655 24490 catalog_manager.cc:5879] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1: Loaded TSK: 0
I20260808 14:32:10.155050 24490 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1: Generated new TSK 1
I20260808 14:32:10.155356 24490 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 14:32:10.211601 24479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:10.211937 24479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:10.212035 24479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:10.216362 24479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:10.216454 24479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:10.216566 24479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:32:10.222117 24479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:37689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=33109
--tserver_master_addrs=127.7.125.126:35021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:10.223359 24479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:10.224570 24479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:10.235706 24505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:10.235723 24506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:10.237123 24479 server_base.cc:1034] running on GCE node
W20260808 14:32:10.238205 24508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:10.238821 24479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:10.239712 24479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:10.240878 24479 hybrid_clock.cc:648] HybridClock initialized: now 1786199530240846 us; error 48 us; skew 500 ppm
Aug 08 14:32:10 dist-test-slave-w206 krb5kdc[23822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199530, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:32:10.244752 24479 init.cc:377] Logged in from keytab as kudu/127.7.125.65@KRBTEST.COM (short username kudu)
I20260808 14:32:10.245857 24479 webserver.cc:460] Webserver started at http://127.7.125.65:33109/ using document root <none> and password file <none>
I20260808 14:32:10.246433 24479 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:10.246508 24479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:10.250624 24479 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:32:10.253085 24515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:10.254611 24479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:32:10.254858 24479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "e9894455cfa84afbbcf278234366d04f"
format_stamp: "Formatted at 2026-08-08 14:31:48 on dist-test-slave-w206"
server_key: "2c46c7f4d559287895aff88a7fc342ec"
server_key_iv: "f5990f533b62e3ba33492e42e8a0ecd3"
server_key_version: "encryptionkey@0"
I20260808 14:32:10.255546 24479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:10.287320 24479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:10.291144 24479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:10.291428 24479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:10.292109 24479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:10.293478 24522 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 14:32:10.298163 24479 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 14:32:10.298301 24479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260808 14:32:10.298394 24479 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 14:32:10.300241 24479 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 14:32:10.300314 24479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260808 14:32:10.300637 24522 tablet_bootstrap.cc:492] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f: Bootstrap starting.
I20260808 14:32:10.311184 24479 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:37689
I20260808 14:32:10.312239 24479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:32:10.313124 24635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:37689 every 8 connection(s)
I20260808 14:32:10.316927 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 24479
I20260808 14:32:10.317497 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:34299
--local_ip_for_outbound_sockets=127.7.125.66
--tserver_master_addrs=127.7.125.126:35021
--webserver_port=33633
--webserver_interface=127.7.125.66
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:32:10.323750 24522 log.cc:824] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:32:10 dist-test-slave-w206 krb5kdc[23822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199530, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:32:10.333155 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:10.314660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.65:56723)
0808 14:32:10.315350 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:10.315418 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:10.316411 (+ 993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:10.318066 (+ 1655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:10.318075 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:10.318510 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:10.319119 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:10.319138 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.321298 (+ 2160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.321302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:10.322742 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:10.322756 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.323812 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.323818 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:10.324173 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:10.325021 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:32:10.325063 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:10.327039 (+ 1976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:10.329568 (+ 2529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:10.329595 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:10.329610 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:10.329937 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:10.330358 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:10.330362 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:10.330365 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:10.330526 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:10.331243 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:10.331250 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:10.331522 (+ 272us) client_negotiation.cc:769] Sending connection context
0808 14:32:10.331727 (+ 205us) client_negotiation.cc:240] Negotiation successful
0808 14:32:10.332121 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":196,"threads_started":1}
I20260808 14:32:10.333137 24638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:10.314598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:56723 (local address 127.7.125.126:35021)
0808 14:32:10.315143 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:10.315166 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:10.316731 (+ 1565us) server_negotiation.cc:407] Connection header received
0808 14:32:10.316985 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:10.316995 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:10.317414 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:10.317800 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:10.319330 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.321084 (+ 1754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.322900 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.323635 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.323891 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:10.327288 (+ 3397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:10.327294 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:10.327309 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:10.327342 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:10.329406 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:10.330087 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:10.330099 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:10.330103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:10.330238 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:10.330641 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:10.330645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:10.330647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:10.331115 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:10.331281 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:10.331873 (+ 592us) server_negotiation.cc:299] Negotiation successful
0808 14:32:10.332167 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":177,"threads_started":1}
I20260808 14:32:10.335517 24522 tablet_bootstrap.cc:492] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:32:10.336045 24522 tablet_bootstrap.cc:492] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f: Bootstrap complete.
I20260808 14:32:10.336304 24636 heartbeater.cc:347] Connected to a master server at 127.7.125.126:35021
I20260808 14:32:10.336582 24636 heartbeater.cc:464] Registering TS with master...
I20260808 14:32:10.337185 24636 heartbeater.cc:511] Master 127.7.125.126:35021 requested a full tablet report, sending...
I20260808 14:32:10.337791 24522 ts_tablet_manager.cc:1397] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f: Time spent bootstrapping tablet: real 0.037s user 0.026s sys 0.008s
I20260808 14:32:10.339860 24436 ts_manager.cc:194] Registered new tserver with Master: e9894455cfa84afbbcf278234366d04f (127.7.125.65:37689)
I20260808 14:32:10.341152 24522 raft_consensus.cc:348] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } } peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } }
I20260808 14:32:10.341759 24522 raft_consensus.cc:729] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9894455cfa84afbbcf278234366d04f, State: Initialized, Role: FOLLOWER
I20260808 14:32:10.342311 24522 consensus_queue.cc:260] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } } peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } }
I20260808 14:32:10.343701 24522 ts_tablet_manager.cc:1428] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f: Time spent starting tablet: real 0.006s user 0.007s sys 0.001s
I20260808 14:32:10.344089 24436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:56723
I20260808 14:32:10.346797 24636 heartbeater.cc:503] Master 127.7.125.126:35021 was elected leader, sending a full tablet report...
Aug 08 14:32:10 dist-test-slave-w206 krb5kdc[23822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199530, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for kudu/127.7.125.65@KRBTEST.COM
I20260808 14:32:10.357112 24644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:10.343284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:37689 (local address 127.0.0.1:41530)
0808 14:32:10.344510 (+ 1226us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:10.344572 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:10.344730 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:10.345323 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:10.345330 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:10.345836 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:10.345988 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:10.345997 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.348590 (+ 2593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.348596 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:10.349825 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:10.349840 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.350478 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.350482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:10.350615 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:10.351332 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:32:10.351354 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:10.353015 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:10.355128 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:10.355140 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:10.355146 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:10.355399 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:10.355745 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:10.355748 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:10.355750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:10.355848 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:10.356399 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:10.356406 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:10.356541 (+ 135us) client_negotiation.cc:769] Sending connection context
0808 14:32:10.356687 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 14:32:10.356922 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1025,"thread_start_us":113,"threads_started":1}
I20260808 14:32:10.357112 24645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:10.344112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41530 (local address 127.7.125.65:37689)
0808 14:32:10.344678 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:10.344691 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:10.344720 (+ 29us) server_negotiation.cc:407] Connection header received
0808 14:32:10.344911 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:10.344920 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:10.345145 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:10.347004 (+ 1859us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:10.347160 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.348434 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.349959 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.350335 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.350429 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:10.353182 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:10.353192 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:10.353207 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:10.353237 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:10.354947 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:10.355524 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:10.355530 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:10.355534 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:10.355635 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:10.355942 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:10.355947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:10.355948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:10.356276 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:10.356425 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:10.356782 (+ 357us) server_negotiation.cc:299] Negotiation successful
0808 14:32:10.356934 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":199,"threads_started":1}
I20260808 14:32:10.358162 24566 tablet_service.cc:1514] Processing DeleteTablet for tablet cb89ef98ea4c4c34b742e64c26035499 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:31:49 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:41530
I20260808 14:32:10.359234 24646 tablet_replica.cc:330] stopping tablet replica
I20260808 14:32:10.359598 24646 raft_consensus.cc:2229] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:32:10.359886 24646 raft_consensus.cc:2258] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:32:10.361159 24646 ts_tablet_manager.cc:1905] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:32:10.366503 24646 ts_tablet_manager.cc:1918] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:32:10.366626 24646 log.cc:1196] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/wal/wals/cb89ef98ea4c4c34b742e64c26035499
I20260808 14:32:10.366954 24646 ts_tablet_manager.cc:1939] T cb89ef98ea4c4c34b742e64c26035499 P e9894455cfa84afbbcf278234366d04f: Deleting consensus metadata
I20260808 14:32:10.369166 24420 catalog_manager.cc:4841] TS e9894455cfa84afbbcf278234366d04f (127.7.125.65:37689): tablet cb89ef98ea4c4c34b742e64c26035499 (table test-table [id=57c469d7ccd944279d32729a3d4d2cec]) successfully deleted
W20260808 14:32:10.458941 24640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:10.459254 24640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:10.459306 24640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:10.463935 24640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:10.464001 24640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:10.464111 24640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:32:10.470126 24640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:34299
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=33633
--tserver_master_addrs=127.7.125.126:35021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:10.471279 24640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:10.472393 24640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:10.482563 24654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:10.482590 24652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:10.482564 24651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:10.483173 24640 server_base.cc:1034] running on GCE node
I20260808 14:32:10.483579 24640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:10.484244 24640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:10.485471 24640 hybrid_clock.cc:648] HybridClock initialized: now 1786199530485452 us; error 33 us; skew 500 ppm
Aug 08 14:32:10 dist-test-slave-w206 krb5kdc[23822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199530, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:32:10.488461 24640 init.cc:377] Logged in from keytab as kudu/127.7.125.66@KRBTEST.COM (short username kudu)
I20260808 14:32:10.489368 24640 webserver.cc:460] Webserver started at http://127.7.125.66:33633/ using document root <none> and password file <none>
I20260808 14:32:10.489907 24640 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:10.489982 24640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:10.494143 24640 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:32:10.496326 24661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:10.497659 24640 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:32:10.497833 24640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "20c8a943104740ae80d2d8a4afd40732"
format_stamp: "Formatted at 2026-08-08 14:31:49 on dist-test-slave-w206"
server_key: "2a0ea25217641f33042ee4b10e4446e4"
server_key_iv: "896e27f5f284d47396fc98248cc5c282"
server_key_version: "encryptionkey@0"
I20260808 14:32:10.498406 24640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:10.513468 24640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:10.517004 24640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:10.517235 24640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:10.517868 24640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:10.518733 24640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:32:10.518783 24640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:10.518846 24640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:32:10.518880 24640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:10.528263 24640 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:34299
I20260808 14:32:10.528335 24784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:34299 every 8 connection(s)
I20260808 14:32:10.529301 24640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:32:10.529552 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 24640
I20260808 14:32:10.530049 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:34405
--local_ip_for_outbound_sockets=127.7.125.67
--tserver_master_addrs=127.7.125.126:35021
--webserver_port=34825
--webserver_interface=127.7.125.67
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 14:32:10 dist-test-slave-w206 krb5kdc[23822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199530, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:32:10.544997 24638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:10.531431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:56401 (local address 127.7.125.126:35021)
0808 14:32:10.531607 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:10.531614 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:10.532538 (+ 924us) server_negotiation.cc:407] Connection header received
0808 14:32:10.533869 (+ 1331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:10.533875 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:10.533946 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:10.534056 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:10.535406 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.536375 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.537590 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.538058 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.538198 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:10.540825 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:10.540832 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:10.540836 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:10.540875 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:10.542775 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:10.543409 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:10.543412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:10.543414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:10.543510 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:10.543840 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:10.543843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:10.543845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:10.544069 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:10.544161 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:10.544674 (+ 513us) server_negotiation.cc:299] Negotiation successful
0808 14:32:10.544823 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 14:32:10.545812 24788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:10.531893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.66:56401)
0808 14:32:10.532364 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:10.532417 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:10.533648 (+ 1231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:10.534212 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:10.534219 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:10.534628 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:10.535236 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:10.535258 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.536530 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.536533 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:10.537457 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:10.537466 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.538217 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.538220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:10.538414 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:10.539082 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:32:10.539099 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:10.540626 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:10.542906 (+ 2280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:10.542911 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:10.542925 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:10.543311 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:10.543603 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:10.543605 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:10.543607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:10.543740 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:10.544162 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:10.544167 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:10.544439 (+ 272us) client_negotiation.cc:769] Sending connection context
0808 14:32:10.544590 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 14:32:10.544844 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":132,"threads_started":1}
I20260808 14:32:10.547154 24785 heartbeater.cc:347] Connected to a master server at 127.7.125.126:35021
I20260808 14:32:10.547412 24785 heartbeater.cc:464] Registering TS with master...
I20260808 14:32:10.548025 24785 heartbeater.cc:511] Master 127.7.125.126:35021 requested a full tablet report, sending...
I20260808 14:32:10.549149 24436 ts_manager.cc:194] Registered new tserver with Master: 20c8a943104740ae80d2d8a4afd40732 (127.7.125.66:34299)
I20260808 14:32:10.549727 24436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:56401
W20260808 14:32:10.646553 24787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:10.646842 24787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:10.646926 24787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:10.651085 24787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:10.651171 24787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:10.651316 24787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:32:10.656670 24787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:34405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=34825
--tserver_master_addrs=127.7.125.126:35021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:10.657759 24787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:10.658805 24787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:10.668639 24796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:10.668639 24793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:10.668635 24794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:10.668907 24787 server_base.cc:1034] running on GCE node
I20260808 14:32:10.669328 24787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:10.669966 24787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:10.671149 24787 hybrid_clock.cc:648] HybridClock initialized: now 1786199530671120 us; error 54 us; skew 500 ppm
Aug 08 14:32:10 dist-test-slave-w206 krb5kdc[23822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199530, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:32:10.674248 24787 init.cc:377] Logged in from keytab as kudu/127.7.125.67@KRBTEST.COM (short username kudu)
I20260808 14:32:10.675287 24787 webserver.cc:460] Webserver started at http://127.7.125.67:34825/ using document root <none> and password file <none>
I20260808 14:32:10.675906 24787 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:10.676029 24787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:10.680320 24787 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:32:10.682519 24803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:10.683889 24787 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:32:10.684033 24787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "909465f4961c4c42b3a2f8876628cd13"
format_stamp: "Formatted at 2026-08-08 14:31:49 on dist-test-slave-w206"
server_key: "63477528bb8f9b7bdc99e2ff0740571a"
server_key_iv: "bcffdad8370fe02935ed5c6620853112"
server_key_version: "encryptionkey@0"
I20260808 14:32:10.684616 24787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:10.695808 24787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:10.698787 24787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:10.699064 24787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:10.699677 24787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:10.700560 24787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:32:10.700637 24787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:10.700724 24787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:32:10.700773 24787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:10.710911 24787 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:34405
I20260808 14:32:10.710940 24923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:34405 every 8 connection(s)
I20260808 14:32:10.711985 24787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:32:10.717916 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 24787
Aug 08 14:32:10 dist-test-slave-w206 krb5kdc[23822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199530, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:32:10.727521 24638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:10.713831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:58833 (local address 127.7.125.126:35021)
0808 14:32:10.713990 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:10.713995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:10.715168 (+ 1173us) server_negotiation.cc:407] Connection header received
0808 14:32:10.716096 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:10.716099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:10.716171 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:10.716302 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:10.717416 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.718247 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.720009 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.720257 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.720350 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:10.722809 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:10.722812 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:10.722815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:10.722839 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:10.724719 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:10.725375 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:10.725379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:10.725381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:10.725479 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:10.726001 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:10.726005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:10.726008 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:10.726414 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:10.726512 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:10.727150 (+ 638us) server_negotiation.cc:299] Negotiation successful
0808 14:32:10.727295 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57,"spinlock_wait_cycles":13056}
I20260808 14:32:10.728977 24926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:10.714308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.67:58833)
0808 14:32:10.714961 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:10.715054 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:10.715891 (+ 837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:10.716466 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:10.716473 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:10.716817 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:10.717274 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:10.717287 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.718399 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.718406 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:10.719854 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:10.719867 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.720426 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.720437 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:10.720582 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:10.721184 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:32:10.721201 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:10.722620 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:10.724847 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:10.724854 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:10.724869 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:10.725270 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:10.725624 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:10.725628 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:10.725659 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:10.725868 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:10.726578 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:10.726583 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:10.726841 (+ 258us) client_negotiation.cc:769] Sending connection context
0808 14:32:10.727099 (+ 258us) client_negotiation.cc:240] Negotiation successful
0808 14:32:10.727481 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":115,"threads_started":1}
I20260808 14:32:10.729678 24927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:10.718747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49084 (local address 127.7.125.126:35021)
0808 14:32:10.719072 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:10.719075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:10.719267 (+ 192us) server_negotiation.cc:407] Connection header received
0808 14:32:10.719378 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:10.719380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:10.719420 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:10.719498 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:10.720434 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.721002 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.721962 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.722262 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.722374 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:10.724032 (+ 1658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:10.724039 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:10.724045 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:10.724157 (+ 112us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:10.726379 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:10.727930 (+ 1551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:10.727934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:10.727936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:10.728032 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:10.728462 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:10.728465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:10.728467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:10.728686 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:10.728907 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:10.729049 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 14:32:10.729184 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":119,"threads_started":1}
I20260808 14:32:10.730548 24924 heartbeater.cc:347] Connected to a master server at 127.7.125.126:35021
I20260808 14:32:10.730813 24924 heartbeater.cc:464] Registering TS with master...
I20260808 14:32:10.731436 24924 heartbeater.cc:511] Master 127.7.125.126:35021 requested a full tablet report, sending...
I20260808 14:32:10.732457 24436 ts_manager.cc:194] Registered new tserver with Master: 909465f4961c4c42b3a2f8876628cd13 (127.7.125.67:34405)
I20260808 14:32:10.733026 24436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:58833
I20260808 14:32:10.744125 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:32:10.748533 24927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:10.744767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49096 (local address 127.7.125.126:35021)
0808 14:32:10.744901 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:10.744904 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:10.745074 (+ 170us) server_negotiation.cc:407] Connection header received
0808 14:32:10.745249 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:10.745252 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:10.745365 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:10.745462 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:32:10.745679 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.746270 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.747241 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.747484 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.747635 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:10.747906 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:32:10.748084 (+ 178us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 14:32:10.748417 (+ 333us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49096: token expired
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 14:32:10.748596 24927 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49096: token expired
W20260808 14:32:10.748934 24357 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.7.125.126:35021)
I20260808 14:32:10.756016 24927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:10.749349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49098 (local address 127.7.125.126:35021)
0808 14:32:10.749455 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:10.749458 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:10.749480 (+ 22us) server_negotiation.cc:407] Connection header received
0808 14:32:10.749509 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:10.749511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:10.749544 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:10.749608 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:10.750190 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.750776 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.751594 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.751824 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.751903 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:10.753208 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:10.753211 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:10.753213 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:10.753237 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:10.754552 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:10.754934 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:10.754937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:10.754938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:10.755081 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:10.755337 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:10.755339 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:10.755340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:10.755511 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:10.755609 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:10.755809 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 14:32:10.755903 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 14:32:10.758579 24436 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49098:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:32:10.762432 24436 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:32:10.770642 24566 tablet_service.cc:1467] Processing CreateTablet for tablet c136be247ba943b8b2ca37edf4f96818 (DEFAULT_TABLE table=test-table [id=9f8fd464f4f74f9a9c205ce390b97b2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:32:10.771360 24566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c136be247ba943b8b2ca37edf4f96818. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:10.774432 24522 tablet_bootstrap.cc:492] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f: Bootstrap starting.
I20260808 14:32:10.774886 24644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:10.769405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:34299 (local address 127.0.0.1:57098)
0808 14:32:10.769953 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:10.769970 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:10.770070 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:10.770667 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:10.770670 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:10.770695 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:10.770831 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:10.770840 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.772121 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.772126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:10.773200 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:10.773211 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.774411 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.774416 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:10.774534 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:10.774594 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 14:32:10.774659 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 14:32:10.774743 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473}
I20260808 14:32:10.775518 24522 tablet_bootstrap.cc:654] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:10.775648 24933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:10.769559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57098 (local address 127.7.125.66:34299)
0808 14:32:10.769905 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:10.769927 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:10.770084 (+ 157us) server_negotiation.cc:407] Connection header received
0808 14:32:10.770194 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:10.770201 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:10.770449 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:10.770696 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:10.770968 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.771987 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.773365 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.774237 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.775111 (+ 874us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:10.775285 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:10.775346 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 14:32:10.775468 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":122,"threads_started":1}
I20260808 14:32:10.775691 24934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:10.769591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:34405 (local address 127.0.0.1:42726)
0808 14:32:10.770044 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:10.770055 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:10.770130 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:10.771210 (+ 1080us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:10.771213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:10.771230 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:10.771340 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:10.771346 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.772856 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.772860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:10.773967 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:10.773979 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.775290 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.775294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:10.775384 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:10.775405 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 14:32:10.775490 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 14:32:10.775564 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":152,"threads_started":1}
I20260808 14:32:10.776477 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:10.770122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42726 (local address 127.7.125.67:34405)
0808 14:32:10.770715 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:10.770726 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:10.770750 (+ 24us) server_negotiation.cc:407] Connection header received
0808 14:32:10.770827 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:10.770834 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:10.771049 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:10.771493 (+ 444us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:10.771662 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.772720 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.774113 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.775166 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.775283 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:10.775419 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:10.775522 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 14:32:10.775609 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":168,"threads_started":1}
I20260808 14:32:10.777772 24522 tablet_bootstrap.cc:492] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f: No bootstrap required, opened a new log
I20260808 14:32:10.777884 24522 ts_tablet_manager.cc:1397] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 14:32:10.778299 24522 raft_consensus.cc:348] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } } peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } }
I20260808 14:32:10.778419 24522 raft_consensus.cc:374] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:10.778466 24522 raft_consensus.cc:729] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9894455cfa84afbbcf278234366d04f, State: Initialized, Role: FOLLOWER
I20260808 14:32:10.778594 24522 consensus_queue.cc:260] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } } peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } }
I20260808 14:32:10.778882 24522 ts_tablet_manager.cc:1428] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 14:32:10.778805 24713 tablet_service.cc:1467] Processing CreateTablet for tablet c136be247ba943b8b2ca37edf4f96818 (DEFAULT_TABLE table=test-table [id=9f8fd464f4f74f9a9c205ce390b97b2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:32:10.779103 24854 tablet_service.cc:1467] Processing CreateTablet for tablet c136be247ba943b8b2ca37edf4f96818 (DEFAULT_TABLE table=test-table [id=9f8fd464f4f74f9a9c205ce390b97b2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:32:10.780340 24713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c136be247ba943b8b2ca37edf4f96818. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:10.780592 24854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c136be247ba943b8b2ca37edf4f96818. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:10.789727 24937 tablet_bootstrap.cc:492] T c136be247ba943b8b2ca37edf4f96818 P 909465f4961c4c42b3a2f8876628cd13: Bootstrap starting.
I20260808 14:32:10.789906 24938 tablet_bootstrap.cc:492] T c136be247ba943b8b2ca37edf4f96818 P 20c8a943104740ae80d2d8a4afd40732: Bootstrap starting.
I20260808 14:32:10.792240 24937 tablet_bootstrap.cc:654] T c136be247ba943b8b2ca37edf4f96818 P 909465f4961c4c42b3a2f8876628cd13: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:10.792650 24938 tablet_bootstrap.cc:654] T c136be247ba943b8b2ca37edf4f96818 P 20c8a943104740ae80d2d8a4afd40732: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:10.793617 24937 log.cc:824] T c136be247ba943b8b2ca37edf4f96818 P 909465f4961c4c42b3a2f8876628cd13: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:10.793617 24938 log.cc:824] T c136be247ba943b8b2ca37edf4f96818 P 20c8a943104740ae80d2d8a4afd40732: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:10.795794 24937 tablet_bootstrap.cc:492] T c136be247ba943b8b2ca37edf4f96818 P 909465f4961c4c42b3a2f8876628cd13: No bootstrap required, opened a new log
I20260808 14:32:10.795799 24938 tablet_bootstrap.cc:492] T c136be247ba943b8b2ca37edf4f96818 P 20c8a943104740ae80d2d8a4afd40732: No bootstrap required, opened a new log
I20260808 14:32:10.796006 24937 ts_tablet_manager.cc:1397] T c136be247ba943b8b2ca37edf4f96818 P 909465f4961c4c42b3a2f8876628cd13: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 14:32:10.796021 24938 ts_tablet_manager.cc:1397] T c136be247ba943b8b2ca37edf4f96818 P 20c8a943104740ae80d2d8a4afd40732: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 14:32:10.798841 24937 raft_consensus.cc:348] T c136be247ba943b8b2ca37edf4f96818 P 909465f4961c4c42b3a2f8876628cd13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } } peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } }
I20260808 14:32:10.798842 24938 raft_consensus.cc:348] T c136be247ba943b8b2ca37edf4f96818 P 20c8a943104740ae80d2d8a4afd40732 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } } peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } }
I20260808 14:32:10.799163 24937 raft_consensus.cc:374] T c136be247ba943b8b2ca37edf4f96818 P 909465f4961c4c42b3a2f8876628cd13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:10.799162 24938 raft_consensus.cc:374] T c136be247ba943b8b2ca37edf4f96818 P 20c8a943104740ae80d2d8a4afd40732 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:10.799226 24937 raft_consensus.cc:729] T c136be247ba943b8b2ca37edf4f96818 P 909465f4961c4c42b3a2f8876628cd13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 909465f4961c4c42b3a2f8876628cd13, State: Initialized, Role: FOLLOWER
I20260808 14:32:10.799263 24938 raft_consensus.cc:729] T c136be247ba943b8b2ca37edf4f96818 P 20c8a943104740ae80d2d8a4afd40732 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c8a943104740ae80d2d8a4afd40732, State: Initialized, Role: FOLLOWER
I20260808 14:32:10.799693 24937 consensus_queue.cc:260] T c136be247ba943b8b2ca37edf4f96818 P 909465f4961c4c42b3a2f8876628cd13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } } peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } }
I20260808 14:32:10.799703 24938 consensus_queue.cc:260] T c136be247ba943b8b2ca37edf4f96818 P 20c8a943104740ae80d2d8a4afd40732 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } } peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } }
I20260808 14:32:10.800436 24924 heartbeater.cc:503] Master 127.7.125.126:35021 was elected leader, sending a full tablet report...
I20260808 14:32:10.800760 24937 ts_tablet_manager.cc:1428] T c136be247ba943b8b2ca37edf4f96818 P 909465f4961c4c42b3a2f8876628cd13: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 14:32:10.800800 24938 ts_tablet_manager.cc:1428] T c136be247ba943b8b2ca37edf4f96818 P 20c8a943104740ae80d2d8a4afd40732: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 14:32:10.801195 24785 heartbeater.cc:503] Master 127.7.125.126:35021 was elected leader, sending a full tablet report...
W20260808 14:32:10.813580 24637 tablet.cc:2369] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:32:10.888361 24643 raft_consensus.cc:482] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:32:10.888650 24643 raft_consensus.cc:504] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } } peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } }
I20260808 14:32:10.889889 24643 leader_election.cc:288] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20c8a943104740ae80d2d8a4afd40732 (127.7.125.66:34299), 909465f4961c4c42b3a2f8876628cd13 (127.7.125.67:34405)
I20260808 14:32:10.894973 24943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:10.890171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:34405 (local address 127.7.125.65:36907)
0808 14:32:10.890538 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:10.890552 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:10.890614 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:10.890914 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:10.890917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:10.890960 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:10.891178 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:10.891186 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.892368 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.892372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:10.893899 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:10.893911 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.894542 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.894545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:10.894616 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:10.894658 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 14:32:10.894726 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 14:32:10.894795 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":107,"spinlock_wait_cycles":7680,"thread_start_us":81,"threads_started":1}
I20260808 14:32:10.895026 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:10.890503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:36907 (local address 127.7.125.67:34405)
0808 14:32:10.890646 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:10.890649 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:10.890677 (+ 28us) server_negotiation.cc:407] Connection header received
0808 14:32:10.890732 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:10.890736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:10.890807 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:10.890885 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:10.891372 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.892171 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.894016 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.894431 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.894530 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:10.894570 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:10.894778 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 14:32:10.894855 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 14:32:10.895794 24944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:10.890085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:34299 (local address 127.7.125.65:45097)
0808 14:32:10.890444 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:10.890461 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:10.890595 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:10.890914 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:10.890917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:10.890959 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:10.891182 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:10.891187 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.892368 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.892372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:10.893902 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:10.893911 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.895408 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.895412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:10.895517 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:10.895536 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 14:32:10.895597 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 14:32:10.895663 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":229,"spinlock_wait_cycles":67328,"thread_start_us":79,"threads_started":1}
I20260808 14:32:10.895700 24874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c136be247ba943b8b2ca37edf4f96818" candidate_uuid: "e9894455cfa84afbbcf278234366d04f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "909465f4961c4c42b3a2f8876628cd13" is_pre_election: true
I20260808 14:32:10.895823 24933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:10.890246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:45097 (local address 127.7.125.66:34299)
0808 14:32:10.890393 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:10.890397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:10.890552 (+ 155us) server_negotiation.cc:407] Connection header received
0808 14:32:10.890734 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:10.890737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:10.890793 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:10.890887 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:10.891365 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.892154 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.894626 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.895276 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.895399 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:10.895454 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:10.895632 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 14:32:10.895710 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 14:32:10.896077 24874 raft_consensus.cc:2454] T c136be247ba943b8b2ca37edf4f96818 P 909465f4961c4c42b3a2f8876628cd13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9894455cfa84afbbcf278234366d04f in term 0.
I20260808 14:32:10.896517 24734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c136be247ba943b8b2ca37edf4f96818" candidate_uuid: "e9894455cfa84afbbcf278234366d04f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c8a943104740ae80d2d8a4afd40732" is_pre_election: true
I20260808 14:32:10.896585 24519 leader_election.cc:302] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 909465f4961c4c42b3a2f8876628cd13, e9894455cfa84afbbcf278234366d04f; no voters:
I20260808 14:32:10.896821 24734 raft_consensus.cc:2454] T c136be247ba943b8b2ca37edf4f96818 P 20c8a943104740ae80d2d8a4afd40732 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9894455cfa84afbbcf278234366d04f in term 0.
I20260808 14:32:10.896886 24643 raft_consensus.cc:2781] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:32:10.896982 24643 raft_consensus.cc:482] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:32:10.897037 24643 raft_consensus.cc:3037] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:10.898192 24643 raft_consensus.cc:504] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } } peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } }
I20260808 14:32:10.898561 24643 leader_election.cc:288] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 1 election: Requested vote from peers 20c8a943104740ae80d2d8a4afd40732 (127.7.125.66:34299), 909465f4961c4c42b3a2f8876628cd13 (127.7.125.67:34405)
I20260808 14:32:10.898870 24734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c136be247ba943b8b2ca37edf4f96818" candidate_uuid: "e9894455cfa84afbbcf278234366d04f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c8a943104740ae80d2d8a4afd40732"
I20260808 14:32:10.898994 24874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c136be247ba943b8b2ca37edf4f96818" candidate_uuid: "e9894455cfa84afbbcf278234366d04f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "909465f4961c4c42b3a2f8876628cd13"
I20260808 14:32:10.899039 24734 raft_consensus.cc:3037] T c136be247ba943b8b2ca37edf4f96818 P 20c8a943104740ae80d2d8a4afd40732 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:10.899155 24874 raft_consensus.cc:3037] T c136be247ba943b8b2ca37edf4f96818 P 909465f4961c4c42b3a2f8876628cd13 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:10.900418 24734 raft_consensus.cc:2454] T c136be247ba943b8b2ca37edf4f96818 P 20c8a943104740ae80d2d8a4afd40732 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9894455cfa84afbbcf278234366d04f in term 1.
I20260808 14:32:10.900418 24874 raft_consensus.cc:2454] T c136be247ba943b8b2ca37edf4f96818 P 909465f4961c4c42b3a2f8876628cd13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9894455cfa84afbbcf278234366d04f in term 1.
I20260808 14:32:10.900743 24517 leader_election.cc:302] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c8a943104740ae80d2d8a4afd40732, e9894455cfa84afbbcf278234366d04f; no voters:
I20260808 14:32:10.900933 24643 raft_consensus.cc:2781] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:32:10.901153 24643 raft_consensus.cc:686] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 1 LEADER]: Becoming Leader. State: Replica: e9894455cfa84afbbcf278234366d04f, State: Running, Role: LEADER
I20260808 14:32:10.901433 24643 consensus_queue.cc:237] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } } peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } }
I20260808 14:32:10.904606 24436 catalog_manager.cc:5495] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f reported cstate change: term changed from 0 to 1, leader changed from <none> to e9894455cfa84afbbcf278234366d04f (127.7.125.65). New cstate: current_term: 1 leader_uuid: "e9894455cfa84afbbcf278234366d04f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } health_report { overall_health: HEALTHY } } }
I20260808 14:32:10.932029 24948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:10.928230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41540 (local address 127.7.125.65:37689)
0808 14:32:10.928558 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:10.928565 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:10.928578 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:32:10.928654 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:10.928656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:10.928709 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:10.928875 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:32:10.929116 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.929727 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.930556 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:10.930814 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:10.930904 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:10.931172 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:32:10.931604 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:32:10.931697 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:10.931771 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 14:32:10.931819 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":148,"threads_started":1}
I20260808 14:32:10.936084 24734 raft_consensus.cc:1261] T c136be247ba943b8b2ca37edf4f96818 P 20c8a943104740ae80d2d8a4afd40732 [term 1 FOLLOWER]: Refusing update from remote peer e9894455cfa84afbbcf278234366d04f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:32:10.936084 24874 raft_consensus.cc:1261] T c136be247ba943b8b2ca37edf4f96818 P 909465f4961c4c42b3a2f8876628cd13 [term 1 FOLLOWER]: Refusing update from remote peer e9894455cfa84afbbcf278234366d04f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:32:10.936830 24643 consensus_queue.cc:1035] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [LEADER]: Connected to new peer: Peer: permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:32:10.937052 24945 consensus_queue.cc:1035] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:32:10.945742 24950 mvcc.cc:205] Tried to move back new op lower bound from 7316273278707445760 to 7316273278575382528. Current Snapshot: MvccSnapshot[applied={T|T < 7316273278707445760}]
I20260808 14:32:10.947021 24951 mvcc.cc:205] Tried to move back new op lower bound from 7316273278707445760 to 7316273278575382528. Current Snapshot: MvccSnapshot[applied={T|T < 7316273278707445760 or (T in {7316273278707445760})}]
I20260808 14:32:10.952365 24434 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:32:10.952533 24434 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:32:10.954814 24434 catalog_manager.cc:5782] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1: Sending DeleteTablet for 3 replicas of tablet c136be247ba943b8b2ca37edf4f96818
I20260808 14:32:10.955551 24566 tablet_service.cc:1514] Processing DeleteTablet for tablet c136be247ba943b8b2ca37edf4f96818 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:32:10 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:41530
I20260808 14:32:10.955651 24854 tablet_service.cc:1514] Processing DeleteTablet for tablet c136be247ba943b8b2ca37edf4f96818 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:32:10 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:42726
I20260808 14:32:10.955653 24713 tablet_service.cc:1514] Processing DeleteTablet for tablet c136be247ba943b8b2ca37edf4f96818 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:32:10 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:57098
I20260808 14:32:10.956149 24963 tablet_replica.cc:330] T c136be247ba943b8b2ca37edf4f96818 P 20c8a943104740ae80d2d8a4afd40732: stopping tablet replica
I20260808 14:32:10.956382 24964 tablet_replica.cc:330] T c136be247ba943b8b2ca37edf4f96818 P 909465f4961c4c42b3a2f8876628cd13: stopping tablet replica
I20260808 14:32:10.956533 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 24479
I20260808 14:32:10.956518 24963 raft_consensus.cc:2229] T c136be247ba943b8b2ca37edf4f96818 P 20c8a943104740ae80d2d8a4afd40732 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:32:10.956799 24964 raft_consensus.cc:2229] T c136be247ba943b8b2ca37edf4f96818 P 909465f4961c4c42b3a2f8876628cd13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:32:10.956864 24963 raft_consensus.cc:2258] T c136be247ba943b8b2ca37edf4f96818 P 20c8a943104740ae80d2d8a4afd40732 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:32:10.957700 24964 raft_consensus.cc:2258] T c136be247ba943b8b2ca37edf4f96818 P 909465f4961c4c42b3a2f8876628cd13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:32:10.959456 24964 ts_tablet_manager.cc:1905] T c136be247ba943b8b2ca37edf4f96818 P 909465f4961c4c42b3a2f8876628cd13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:32:10.959457 24963 ts_tablet_manager.cc:1905] T c136be247ba943b8b2ca37edf4f96818 P 20c8a943104740ae80d2d8a4afd40732: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:32:10.962155 24963 ts_tablet_manager.cc:1918] T c136be247ba943b8b2ca37edf4f96818 P 20c8a943104740ae80d2d8a4afd40732: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:32:10.962375 24963 log.cc:1196] T c136be247ba943b8b2ca37edf4f96818 P 20c8a943104740ae80d2d8a4afd40732: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/c136be247ba943b8b2ca37edf4f96818
I20260808 14:32:10.962797 24963 ts_tablet_manager.cc:1939] T c136be247ba943b8b2ca37edf4f96818 P 20c8a943104740ae80d2d8a4afd40732: Deleting consensus metadata
I20260808 14:32:10.963091 24964 ts_tablet_manager.cc:1918] T c136be247ba943b8b2ca37edf4f96818 P 909465f4961c4c42b3a2f8876628cd13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:32:10.963198 24964 log.cc:1196] T c136be247ba943b8b2ca37edf4f96818 P 909465f4961c4c42b3a2f8876628cd13: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/c136be247ba943b8b2ca37edf4f96818
I20260808 14:32:10.963541 24964 ts_tablet_manager.cc:1939] T c136be247ba943b8b2ca37edf4f96818 P 909465f4961c4c42b3a2f8876628cd13: Deleting consensus metadata
I20260808 14:32:10.964478 24419 catalog_manager.cc:4841] TS 20c8a943104740ae80d2d8a4afd40732 (127.7.125.66:34299): tablet c136be247ba943b8b2ca37edf4f96818 (table test-table [id=9f8fd464f4f74f9a9c205ce390b97b2a]) successfully deleted
I20260808 14:32:10.964771 24421 catalog_manager.cc:4841] TS 909465f4961c4c42b3a2f8876628cd13 (127.7.125.67:34405): tablet c136be247ba943b8b2ca37edf4f96818 (table test-table [id=9f8fd464f4f74f9a9c205ce390b97b2a]) successfully deleted
W20260808 14:32:10.966248 24420 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.125.65:37689 (error 108)
I20260808 14:32:10.966701 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 24640
I20260808 14:32:10.968518 24934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:10.967012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:37689 (local address 127.0.0.1:41550)
0808 14:32:10.968317 (+ 1305us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:10.968408 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.65:37689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1175}
W20260808 14:32:10.968731 24420 catalog_manager.cc:4584] TS e9894455cfa84afbbcf278234366d04f (127.7.125.65:37689): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c136be247ba943b8b2ca37edf4f96818: Network error: Client connection negotiation failed: client connection to 127.7.125.65:37689: connect: Connection refused (error 111)
I20260808 14:32:10.974139 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 24787
I20260808 14:32:10.981175 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 24400
I20260808 14:32:30.989735 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:35021
--webserver_interface=127.7.125.126
--webserver_port=41987
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:35021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:32:31.104110 24966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:31.104437 24966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:31.104526 24966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:31.108546 24966 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 14:32:31.108624 24966 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 14:32:31.108652 24966 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 14:32:31.108683 24966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:32:31.108728 24966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:31.108783 24966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:31.108821 24966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:32:31.108857 24966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:32:31.114207 24966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:35021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:35021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=41987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:31.115417 24966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:31.116498 24966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:31.125531 24974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:31.125655 24966 server_base.cc:1034] running on GCE node
W20260808 14:32:31.125655 24972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:31.125676 24971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:31.126368 24966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:31.127579 24966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:31.128760 24966 hybrid_clock.cc:648] HybridClock initialized: now 1786199551128740 us; error 33 us; skew 500 ppm
Aug 08 14:32:31 dist-test-slave-w206 krb5kdc[23822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199551, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:32:31.138497 24966 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:32:31.139668 24966 webserver.cc:460] Webserver started at http://127.7.125.126:41987/ using document root <none> and password file <none>
I20260808 14:32:31.140347 24966 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:31.140422 24966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:31.144845 24966 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 14:32:31.147289 24981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:31.148733 24966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 14:32:31.148908 24966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "df1f026924554d6296c793aa96c6efc1"
format_stamp: "Formatted at 2026-08-08 14:31:48 on dist-test-slave-w206"
server_key: "4c99631163c88b253cbefcde2f5bc376"
server_key_iv: "1913ccbab000310daea50975dae2ff2a"
server_key_version: "encryptionkey@0"
I20260808 14:32:31.149526 24966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:31.161525 24966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:31.165050 24966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:31.165302 24966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:31.172873 24966 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:35021
I20260808 14:32:31.173182 25035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:35021 every 8 connection(s)
I20260808 14:32:31.174072 24966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:32:31.178135 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 24966
I20260808 14:32:31.178656 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:37689
--local_ip_for_outbound_sockets=127.7.125.65
--tserver_master_addrs=127.7.125.126:35021
--webserver_port=33109
--webserver_interface=127.7.125.65
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:32:31.178898 25036 sys_catalog.cc:263] Verifying existing consensus state
I20260808 14:32:31.181497 25036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1: Bootstrap starting.
I20260808 14:32:31.200134 25036 log.cc:824] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:31.211694 25036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:32:31.212128 25036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1: Bootstrap complete.
I20260808 14:32:31.216537 25036 raft_consensus.cc:348] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f026924554d6296c793aa96c6efc1" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35021 } }
I20260808 14:32:31.216949 25036 raft_consensus.cc:729] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: df1f026924554d6296c793aa96c6efc1, State: Initialized, Role: FOLLOWER
I20260808 14:32:31.217476 25036 consensus_queue.cc:260] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f026924554d6296c793aa96c6efc1" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35021 } }
I20260808 14:32:31.217595 25036 raft_consensus.cc:388] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:32:31.217689 25036 raft_consensus.cc:482] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:32:31.217813 25036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 2 FOLLOWER]: Advancing to term 3
I20260808 14:32:31.219841 25036 raft_consensus.cc:504] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f026924554d6296c793aa96c6efc1" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35021 } }
I20260808 14:32:31.220211 25036 leader_election.cc:302] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df1f026924554d6296c793aa96c6efc1; no voters:
I20260808 14:32:31.220575 25036 leader_election.cc:288] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 14:32:31.220672 25040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 14:32:31.220877 25040 raft_consensus.cc:686] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [term 3 LEADER]: Becoming Leader. State: Replica: df1f026924554d6296c793aa96c6efc1, State: Running, Role: LEADER
I20260808 14:32:31.221235 25040 consensus_queue.cc:237] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f026924554d6296c793aa96c6efc1" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35021 } }
I20260808 14:32:31.221863 25036 sys_catalog.cc:564] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:32:31.222767 25041 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "df1f026924554d6296c793aa96c6efc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f026924554d6296c793aa96c6efc1" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35021 } } }
I20260808 14:32:31.222894 25041 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [sys.catalog]: This master's current role is: LEADER
I20260808 14:32:31.223311 25042 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df1f026924554d6296c793aa96c6efc1. Latest consensus state: current_term: 3 leader_uuid: "df1f026924554d6296c793aa96c6efc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1f026924554d6296c793aa96c6efc1" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 35021 } } }
I20260808 14:32:31.223426 25042 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1 [sys.catalog]: This master's current role is: LEADER
I20260808 14:32:31.226846 25056 catalog_manager.cc:1259] Loaded cluster ID: d3d1526fb6364165bb3538f7cb0b0049
I20260808 14:32:31.226938 25056 catalog_manager.cc:1552] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1: loading cluster ID for follower catalog manager: success
I20260808 14:32:31.228744 25056 catalog_manager.cc:1574] T 00000000000000000000000000000000 P df1f026924554d6296c793aa96c6efc1: acquiring CA information for follower catalog manager: success
F20260808 14:32:31.229712 25056 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f8e71a839ad google::LogMessage::Fail() at ??:0
@ 0x7f8e71a8688a google::LogMessage::SendToLog() at ??:0
@ 0x7f8e71a834de google::LogMessage::Flush() at ??:0
@ 0x7f8e71a84a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8e7474718d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f8e7c3e8989 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7f8e7c3c5023 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7f8e7c3c554b kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7f8e7c3be652 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7f8e7c3bd9e9 _ZZN4kudu6master21CatalogManagerBgTasks4InitEvENKUlvE_clEv at ??:0
@ 0x7f8e7c404541 _ZNSt17_Function_handlerIFvvEZN4kudu6master21CatalogManagerBgTasks4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f8e73cc6512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f8e7530a6db start_thread at ??:0
@ 0x7f8e7058161f clone at ??:0
W20260808 14:32:31.315605 25038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:31.315905 25038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:31.315980 25038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:31.320151 25038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:31.320238 25038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:31.320331 25038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:32:31.325894 25038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:37689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=33109
--tserver_master_addrs=127.7.125.126:35021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:31.327108 25038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:31.328189 25038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:31.338457 25084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:31.338457 25080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:31.338809 25081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:31.338647 25038 server_base.cc:1034] running on GCE node
I20260808 14:32:31.339372 25038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:31.340240 25038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:31.341455 25038 hybrid_clock.cc:648] HybridClock initialized: now 1786199551341430 us; error 34 us; skew 500 ppm
Aug 08 14:32:31 dist-test-slave-w206 krb5kdc[23822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199551, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:32:31.344980 25038 init.cc:377] Logged in from keytab as kudu/127.7.125.65@KRBTEST.COM (short username kudu)
I20260808 14:32:31.346114 25038 webserver.cc:460] Webserver started at http://127.7.125.65:33109/ using document root <none> and password file <none>
I20260808 14:32:31.346729 25038 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:31.346832 25038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:31.351333 25038 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 14:32:31.353631 25092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:31.355063 25038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:32:31.355204 25038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "e9894455cfa84afbbcf278234366d04f"
format_stamp: "Formatted at 2026-08-08 14:31:48 on dist-test-slave-w206"
server_key: "2c46c7f4d559287895aff88a7fc342ec"
server_key_iv: "f5990f533b62e3ba33492e42e8a0ecd3"
server_key_version: "encryptionkey@0"
I20260808 14:32:31.355773 25038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:31.373238 25038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:31.376627 25038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:31.376842 25038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:31.377475 25038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:31.378671 25100 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 14:32:31.384243 25038 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 14:32:31.384328 25038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260808 14:32:31.384433 25038 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 14:32:31.386134 25038 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 14:32:31.386209 25038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 14:32:31.386525 25100 tablet_bootstrap.cc:492] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f: Bootstrap starting.
I20260808 14:32:31.396729 25038 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:37689
I20260808 14:32:31.397820 25038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:32:31.399255 25212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:37689 every 8 connection(s)
I20260808 14:32:31.399345 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 25038
I20260808 14:32:31.399842 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:34299
--local_ip_for_outbound_sockets=127.7.125.66
--tserver_master_addrs=127.7.125.126:35021
--webserver_port=33633
--webserver_interface=127.7.125.66
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:32:31.409370 25100 log.cc:824] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:31.418900 25100 tablet_bootstrap.cc:492] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:32:31.419400 25100 tablet_bootstrap.cc:492] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f: Bootstrap complete.
I20260808 14:32:31.420969 25100 ts_tablet_manager.cc:1397] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f: Time spent bootstrapping tablet: real 0.035s user 0.018s sys 0.014s
I20260808 14:32:31.424329 25100 raft_consensus.cc:348] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } } peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } }
I20260808 14:32:31.424898 25100 raft_consensus.cc:729] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9894455cfa84afbbcf278234366d04f, State: Initialized, Role: FOLLOWER
I20260808 14:32:31.425446 25100 consensus_queue.cc:260] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } } peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } }
I20260808 14:32:31.426539 25100 ts_tablet_manager.cc:1428] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f: Time spent starting tablet: real 0.005s user 0.002s sys 0.005s
W20260808 14:32:31.465199 25216 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:32:31.401650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.65:59945)
0808 14:32:31.408486 (+ 6836us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:31.408579 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:31.409408 (+ 829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:31.464359 (+ 54951us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":6490,"thread_start_us":193,"threads_started":1}
W20260808 14:32:31.465802 25213 heartbeater.cc:644] Failed to heartbeat to 127.7.125.126:35021 (0 consecutive failures): Network error: Failed to ping master at 127.7.125.126:35021: Client connection negotiation failed: client connection to 127.7.125.126:35021: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 14:32:31.466586 25216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:31.466326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.65:47977)
0808 14:32:31.466455 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:31.466531 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 14:32:31.467388 25216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:31.467170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.65:54047)
0808 14:32:31.467273 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:31.467325 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 14:32:31.467588 25213 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 14:32:31.521617 25215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:31.521903 25215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:31.521950 25215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:31.526008 25215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:31.526069 25215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:31.526163 25215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:32:31.531461 25215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:34299
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=33633
--tserver_master_addrs=127.7.125.126:35021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:31.532553 25215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:31.533599 25215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:31.543609 25226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:31.543707 25215 server_base.cc:1034] running on GCE node
W20260808 14:32:31.543634 25223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:31.543599 25224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:31.544297 25215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:31.544940 25215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:31.546104 25215 hybrid_clock.cc:648] HybridClock initialized: now 1786199551546082 us; error 35 us; skew 500 ppm
Aug 08 14:32:31 dist-test-slave-w206 krb5kdc[23822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199551, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:32:31.549198 25215 init.cc:377] Logged in from keytab as kudu/127.7.125.66@KRBTEST.COM (short username kudu)
I20260808 14:32:31.550165 25215 webserver.cc:460] Webserver started at http://127.7.125.66:33633/ using document root <none> and password file <none>
I20260808 14:32:31.550740 25215 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:31.550832 25215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:31.554868 25215 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260808 14:32:31.557022 25234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:31.558290 25215 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:32:31.558429 25215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "20c8a943104740ae80d2d8a4afd40732"
format_stamp: "Formatted at 2026-08-08 14:31:49 on dist-test-slave-w206"
server_key: "2a0ea25217641f33042ee4b10e4446e4"
server_key_iv: "896e27f5f284d47396fc98248cc5c282"
server_key_version: "encryptionkey@0"
I20260808 14:32:31.559020 25215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:31.586532 25215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:31.590008 25215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:31.590233 25215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:31.590873 25215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:31.591753 25215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:32:31.591825 25215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:31.591907 25215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:32:31.591950 25215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:31.602483 25215 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:34299
I20260808 14:32:31.602507 25354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:34299 every 8 connection(s)
I20260808 14:32:31.603690 25215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:32:31.607554 25357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:31.605916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.66:55143)
0808 14:32:31.606603 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:31.606726 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":112,"threads_started":1}
W20260808 14:32:31.608060 25355 heartbeater.cc:644] Failed to heartbeat to 127.7.125.126:35021 (0 consecutive failures): Network error: Failed to ping master at 127.7.125.126:35021: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
I20260808 14:32:31.608620 25357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:31.608410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.66:58063)
0808 14:32:31.608520 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:31.608569 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 14:32:31.609196 25357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:31.609018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.66:44187)
0808 14:32:31.609113 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:31.609152 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 14:32:31.609426 25355 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 14:32:31.609458 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 25215
I20260808 14:32:31.609886 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:34405
--local_ip_for_outbound_sockets=127.7.125.67
--tserver_master_addrs=127.7.125.126:35021
--webserver_port=34825
--webserver_interface=127.7.125.67
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:32:31.723541 25358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:31.723856 25358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:31.723949 25358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:31.728051 25358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:31.728236 25358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:31.728374 25358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:32:31.733948 25358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:34405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=34825
--tserver_master_addrs=127.7.125.126:35021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:31.735174 25358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:31.736204 25358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:31.746044 25364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:31.746088 25363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:31.746042 25366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:31.746515 25358 server_base.cc:1034] running on GCE node
I20260808 14:32:31.746913 25358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:31.747612 25358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:31.748788 25358 hybrid_clock.cc:648] HybridClock initialized: now 1786199551748767 us; error 31 us; skew 500 ppm
Aug 08 14:32:31 dist-test-slave-w206 krb5kdc[23822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199551, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:32:31.752007 25358 init.cc:377] Logged in from keytab as kudu/127.7.125.67@KRBTEST.COM (short username kudu)
I20260808 14:32:31.752918 25358 webserver.cc:460] Webserver started at http://127.7.125.67:34825/ using document root <none> and password file <none>
I20260808 14:32:31.753479 25358 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:31.753544 25358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:31.757578 25358 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 14:32:31.759897 25373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:31.761222 25358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:32:31.761348 25358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "909465f4961c4c42b3a2f8876628cd13"
format_stamp: "Formatted at 2026-08-08 14:31:49 on dist-test-slave-w206"
server_key: "63477528bb8f9b7bdc99e2ff0740571a"
server_key_iv: "bcffdad8370fe02935ed5c6620853112"
server_key_version: "encryptionkey@0"
I20260808 14:32:31.761989 25358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:31.773953 25358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:31.777360 25358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:31.777559 25358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:31.778141 25358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:31.779011 25358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:32:31.779088 25358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:31.779158 25358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:32:31.779215 25358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:31.788111 25358 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:34405
I20260808 14:32:31.788133 25490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:34405 every 8 connection(s)
I20260808 14:32:31.789218 25358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:32:31.793182 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:31.791423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.67:48175)
0808 14:32:31.792168 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:31.792323 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":127,"threads_started":1}
W20260808 14:32:31.793648 25491 heartbeater.cc:644] Failed to heartbeat to 127.7.125.126:35021 (0 consecutive failures): Network error: Failed to ping master at 127.7.125.126:35021: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
I20260808 14:32:31.794428 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:31.794107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.67:58929)
0808 14:32:31.794297 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:31.794360 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 14:32:31.795094 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:31.794843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.67:46003)
0808 14:32:31.794949 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:31.795044 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 14:32:31.795261 25491 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 14:32:31.797533 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 25358
I20260808 14:32:32.468701 25495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:32.468223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.65:53353)
0808 14:32:32.468515 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:32.468582 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":85,"threads_started":1}
I20260808 14:32:32.590526 25496 raft_consensus.cc:482] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:32:32.590744 25496 raft_consensus.cc:504] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } } peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } }
I20260808 14:32:32.592249 25496 leader_election.cc:288] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20c8a943104740ae80d2d8a4afd40732 (127.7.125.66:34299), 909465f4961c4c42b3a2f8876628cd13 (127.7.125.67:34405)
Aug 08 14:32:32 dist-test-slave-w206 krb5kdc[23822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199551, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.66@KRBTEST.COM
Aug 08 14:32:32 dist-test-slave-w206 krb5kdc[23822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199551, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.67@KRBTEST.COM
I20260808 14:32:32.609591 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:32.592458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:34405 (local address 127.7.125.65:49389)
0808 14:32:32.593069 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:32.593084 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:32.593196 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:32.595091 (+ 1895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:32.595098 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:32.595804 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:32.596605 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:32.596628 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:32.598558 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:32.598562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:32.600182 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:32.600195 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:32.601175 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:32.601179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:32.601456 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:32.602849 (+ 1393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:32:32.602955 (+ 106us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:32.604652 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:32.607147 (+ 2495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:32.607153 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:32.607157 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:32.607395 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:32.607737 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:32.607738 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:32.607739 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:32.607815 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:32.608452 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:32.608454 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:32.608800 (+ 346us) client_negotiation.cc:769] Sending connection context
0808 14:32:32.609010 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 14:32:32.609438 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":817,"spinlock_wait_cycles":35328,"thread_start_us":79,"threads_started":1}
I20260808 14:32:32.609663 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:32.592843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:54127 (local address 127.7.125.66:34299)
0808 14:32:32.593194 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:32.593204 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:32.593235 (+ 31us) server_negotiation.cc:407] Connection header received
0808 14:32:32.594092 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:32.594100 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:32.594619 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:32.595007 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:32.596808 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:32.598366 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:32.600319 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:32.601037 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:32.601210 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:32.604448 (+ 3238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:32.604455 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:32.604471 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:32.604506 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:32.606514 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:32.607099 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:32.607107 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:32.607114 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:32.607291 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:32.607719 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:32.607723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:32.607729 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:32.608283 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:32.608432 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:32.609170 (+ 738us) server_negotiation.cc:299] Negotiation successful
0808 14:32:32.609474 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":112,"threads_started":1}
I20260808 14:32:32.609664 25497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:32.592743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:49389 (local address 127.7.125.67:34405)
0808 14:32:32.593145 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:32.593161 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:32.593212 (+ 51us) server_negotiation.cc:407] Connection header received
0808 14:32:32.594090 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:32.594101 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:32.594619 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:32.595002 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:32.596807 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:32.598355 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:32.600320 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:32.601037 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:32.601210 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:32.604874 (+ 3664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:32.604884 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:32.604900 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:32.604939 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:32.607040 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:32.607503 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:32.607513 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:32.607517 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:32.607631 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:32.607914 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:32.607917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:32.607918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:32.608339 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:32.608468 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:32.609170 (+ 702us) server_negotiation.cc:299] Negotiation successful
0808 14:32:32.609481 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":115,"threads_started":1}
I20260808 14:32:32.610579 25495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:32.592458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:34299 (local address 127.7.125.65:54127)
0808 14:32:32.592599 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:32.592618 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:32.592739 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:32.595091 (+ 2352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:32.595098 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:32.595804 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:32.596605 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:32.596628 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:32.598558 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:32.598562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:32.600182 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:32.600195 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:32.601175 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:32.601179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:32.601458 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:32.602295 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:32:32.602319 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:32.604198 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:32.606649 (+ 2451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:32.606655 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:32.606667 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:32.606939 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:32.607424 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:32.607429 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:32.607431 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:32.607601 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:32.608424 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:32.608432 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:32.608800 (+ 368us) client_negotiation.cc:769] Sending connection context
0808 14:32:32.609010 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 14:32:32.609382 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":350,"spinlock_wait_cycles":38656}
I20260808 14:32:32.610728 25501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:32.610317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.66:42819)
0808 14:32:32.610544 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:32.610605 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":91,"threads_started":1}
W20260808 14:32:32.612506 25096 leader_election.cc:341] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 909465f4961c4c42b3a2f8876628cd13 (127.7.125.67:34405): Not found: Tablet not found: c136be247ba943b8b2ca37edf4f96818
W20260808 14:32:32.612774 25094 leader_election.cc:341] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20c8a943104740ae80d2d8a4afd40732 (127.7.125.66:34299): Not found: Tablet not found: c136be247ba943b8b2ca37edf4f96818
I20260808 14:32:32.612874 25094 leader_election.cc:302] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9894455cfa84afbbcf278234366d04f; no voters: 20c8a943104740ae80d2d8a4afd40732, 909465f4961c4c42b3a2f8876628cd13
I20260808 14:32:32.613171 25496 raft_consensus.cc:2726] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 14:32:32.796185 25502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:32.795770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.67:52461)
0808 14:32:32.796030 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:32.796089 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":88,"threads_started":1}
I20260808 14:32:33.470017 25503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:33.469559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.65:49447)
0808 14:32:33.469844 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:33.469918 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":106,"threads_started":1}
I20260808 14:32:33.611970 25504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:33.611537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.66:51779)
0808 14:32:33.611786 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:33.611846 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":79,"threads_started":1}
I20260808 14:32:33.797369 25505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:33.796931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.67:58721)
0808 14:32:33.797182 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:33.797243 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":71,"threads_started":1}
I20260808 14:32:34.471325 25506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:34.470888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.65:37123)
0808 14:32:34.471164 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:34.471224 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":99,"threads_started":1}
I20260808 14:32:34.596354 25507 raft_consensus.cc:482] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:32:34.596504 25507 raft_consensus.cc:504] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } } peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } }
I20260808 14:32:34.596982 25507 leader_election.cc:288] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20c8a943104740ae80d2d8a4afd40732 (127.7.125.66:34299), 909465f4961c4c42b3a2f8876628cd13 (127.7.125.67:34405)
W20260808 14:32:34.597837 25096 leader_election.cc:341] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 909465f4961c4c42b3a2f8876628cd13 (127.7.125.67:34405): Not found: Tablet not found: c136be247ba943b8b2ca37edf4f96818
W20260808 14:32:34.597944 25094 leader_election.cc:341] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20c8a943104740ae80d2d8a4afd40732 (127.7.125.66:34299): Not found: Tablet not found: c136be247ba943b8b2ca37edf4f96818
I20260808 14:32:34.598007 25094 leader_election.cc:302] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9894455cfa84afbbcf278234366d04f; no voters: 20c8a943104740ae80d2d8a4afd40732, 909465f4961c4c42b3a2f8876628cd13
I20260808 14:32:34.598140 25507 raft_consensus.cc:2726] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 14:32:34.613175 25508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:34.612776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.66:60619)
0808 14:32:34.613026 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:34.613085 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":88,"threads_started":1}
I20260808 14:32:34.798702 25509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:34.798231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.67:58327)
0808 14:32:34.798524 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:34.798592 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":91,"threads_started":1}
I20260808 14:32:35.472630 25510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:35.472150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.65:38769)
0808 14:32:35.472432 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:35.472528 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":79,"threads_started":1}
I20260808 14:32:35.614328 25511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:35.613939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.66:43889)
0808 14:32:35.614180 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:35.614235 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":77,"threads_started":1}
I20260808 14:32:35.800040 25512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:35.799612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.67:37071)
0808 14:32:35.799876 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:35.799945 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":102,"threads_started":1}
I20260808 14:32:36.313899 25513 raft_consensus.cc:482] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:32:36.314049 25513 raft_consensus.cc:504] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } } peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } }
I20260808 14:32:36.314471 25513 leader_election.cc:288] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20c8a943104740ae80d2d8a4afd40732 (127.7.125.66:34299), 909465f4961c4c42b3a2f8876628cd13 (127.7.125.67:34405)
W20260808 14:32:36.315464 25094 leader_election.cc:341] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20c8a943104740ae80d2d8a4afd40732 (127.7.125.66:34299): Not found: Tablet not found: c136be247ba943b8b2ca37edf4f96818
W20260808 14:32:36.315578 25096 leader_election.cc:341] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 909465f4961c4c42b3a2f8876628cd13 (127.7.125.67:34405): Not found: Tablet not found: c136be247ba943b8b2ca37edf4f96818
I20260808 14:32:36.315655 25096 leader_election.cc:302] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9894455cfa84afbbcf278234366d04f; no voters: 20c8a943104740ae80d2d8a4afd40732, 909465f4961c4c42b3a2f8876628cd13
I20260808 14:32:36.315801 25513 raft_consensus.cc:2726] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 14:32:36.473935 25514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:36.473499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.65:42605)
0808 14:32:36.473786 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:36.473839 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":75,"threads_started":1}
I20260808 14:32:36.615820 25515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:36.615332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.66:54027)
0808 14:32:36.615642 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:36.615724 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":136,"threads_started":1}
I20260808 14:32:36.801349 25517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:36.800908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.67:47191)
0808 14:32:36.801178 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:36.801251 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":79,"threads_started":1}
I20260808 14:32:37.475220 25518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:37.474712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.65:46555)
0808 14:32:37.475053 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:37.475118 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":74,"threads_started":1}
I20260808 14:32:37.617084 25519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:37.616635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.66:38869)
0808 14:32:37.616924 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:37.616985 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":81,"threads_started":1}
I20260808 14:32:37.802659 25520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:37.802211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.67:43647)
0808 14:32:37.802489 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:37.802556 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":111,"threads_started":1}
I20260808 14:32:38.476567 25522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:38.476121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.65:47091)
0808 14:32:38.476400 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:38.476469 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":110,"threads_started":1}
I20260808 14:32:38.618479 25523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:38.617994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.66:41237)
0808 14:32:38.618308 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:38.618377 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":80,"threads_started":1}
I20260808 14:32:38.804006 25524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:38.803493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.67:52139)
0808 14:32:38.803835 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:38.803912 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":74,"threads_started":1}
I20260808 14:32:39.478042 25525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:39.477574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.65:53199)
0808 14:32:39.477863 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:39.477936 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":86,"threads_started":1}
I20260808 14:32:39.619954 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:39.619467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.66:39857)
0808 14:32:39.619765 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:39.619830 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":94,"threads_started":1}
I20260808 14:32:39.805200 25527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:39.804764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.67:45533)
0808 14:32:39.805040 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:39.805104 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":93,"threads_started":1}
I20260808 14:32:40.479439 25528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:40.478959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.65:51327)
0808 14:32:40.479274 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:40.479339 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":96,"threads_started":1}
I20260808 14:32:40.621179 25530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:40.620739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.66:44083)
0808 14:32:40.621018 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:40.621081 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":82,"threads_started":1}
I20260808 14:32:40.676294 25531 raft_consensus.cc:482] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:32:40.676460 25531 raft_consensus.cc:504] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } } peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } }
I20260808 14:32:40.676942 25531 leader_election.cc:288] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20c8a943104740ae80d2d8a4afd40732 (127.7.125.66:34299), 909465f4961c4c42b3a2f8876628cd13 (127.7.125.67:34405)
W20260808 14:32:40.677857 25094 leader_election.cc:341] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20c8a943104740ae80d2d8a4afd40732 (127.7.125.66:34299): Not found: Tablet not found: c136be247ba943b8b2ca37edf4f96818
W20260808 14:32:40.677966 25096 leader_election.cc:341] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 909465f4961c4c42b3a2f8876628cd13 (127.7.125.67:34405): Not found: Tablet not found: c136be247ba943b8b2ca37edf4f96818
I20260808 14:32:40.678037 25096 leader_election.cc:302] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9894455cfa84afbbcf278234366d04f; no voters: 20c8a943104740ae80d2d8a4afd40732, 909465f4961c4c42b3a2f8876628cd13
I20260808 14:32:40.678218 25531 raft_consensus.cc:2726] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 14:32:40.806509 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:40.806036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.67:49823)
0808 14:32:40.806322 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:40.806405 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":104,"threads_started":1}
I20260808 14:32:41.481037 25533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:41.480435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.65:37079)
0808 14:32:41.480771 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:41.480877 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":147,"threads_started":1}
I20260808 14:32:41.622502 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:41.622055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.66:33573)
0808 14:32:41.622335 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:41.622401 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":75,"threads_started":1}
I20260808 14:32:41.807955 25535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:41.807480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.67:32825)
0808 14:32:41.807774 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:41.807842 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":75,"threads_started":1}
I20260808 14:32:42.482297 25536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:42.481891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.65:52403)
0808 14:32:42.482146 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:42.482202 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":83,"threads_started":1}
I20260808 14:32:42.623914 25537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:42.623441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.66:41883)
0808 14:32:42.623713 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:42.623772 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":104,"threads_started":1}
I20260808 14:32:42.809226 25538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:42.808784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.67:50961)
0808 14:32:42.809060 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:42.809122 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":78,"threads_started":1}
I20260808 14:32:42.921962 25539 raft_consensus.cc:482] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:32:42.922142 25539 raft_consensus.cc:504] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } } peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } }
I20260808 14:32:42.922571 25539 leader_election.cc:288] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20c8a943104740ae80d2d8a4afd40732 (127.7.125.66:34299), 909465f4961c4c42b3a2f8876628cd13 (127.7.125.67:34405)
W20260808 14:32:42.923573 25094 leader_election.cc:341] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20c8a943104740ae80d2d8a4afd40732 (127.7.125.66:34299): Not found: Tablet not found: c136be247ba943b8b2ca37edf4f96818
W20260808 14:32:42.923683 25096 leader_election.cc:341] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 909465f4961c4c42b3a2f8876628cd13 (127.7.125.67:34405): Not found: Tablet not found: c136be247ba943b8b2ca37edf4f96818
I20260808 14:32:42.923754 25096 leader_election.cc:302] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9894455cfa84afbbcf278234366d04f; no voters: 20c8a943104740ae80d2d8a4afd40732, 909465f4961c4c42b3a2f8876628cd13
I20260808 14:32:42.923902 25539 raft_consensus.cc:2726] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 14:32:43.483538 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:43.483136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.65:52035)
0808 14:32:43.483394 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:43.483446 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":76,"threads_started":1}
I20260808 14:32:43.625272 25542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:43.624820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.66:42307)
0808 14:32:43.625069 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:43.625125 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":81,"threads_started":1}
I20260808 14:32:43.810724 25543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:43.810289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.67:47285)
0808 14:32:43.810561 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:43.810625 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":79,"threads_started":1}
I20260808 14:32:44.484773 25544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:44.484363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.65:44247)
0808 14:32:44.484621 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:44.484677 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":84,"threads_started":1}
I20260808 14:32:44.626713 25545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:44.626242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.66:44489)
0808 14:32:44.626535 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:44.626609 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":123,"threads_started":1}
I20260808 14:32:44.812173 25546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:44.811639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.67:41109)
0808 14:32:44.812011 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:44.812076 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":171,"threads_started":1}
I20260808 14:32:45.486176 25547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:45.485717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.65:38073)
0808 14:32:45.486018 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:45.486080 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":103,"threads_started":1}
I20260808 14:32:45.628103 25548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:45.627639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.66:43713)
0808 14:32:45.627941 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:45.628004 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":116,"threads_started":1}
I20260808 14:32:45.813815 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:45.813212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.67:52051)
0808 14:32:45.813632 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:45.813706 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":146,"threads_started":1}
I20260808 14:32:46.487744 25550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:46.487304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.65:39765)
0808 14:32:46.487589 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:46.487648 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":83,"threads_started":1}
I20260808 14:32:46.600291 25551 raft_consensus.cc:482] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:32:46.600473 25551 raft_consensus.cc:504] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c8a943104740ae80d2d8a4afd40732" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34299 } } peers { permanent_uuid: "909465f4961c4c42b3a2f8876628cd13" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 34405 } } peers { permanent_uuid: "e9894455cfa84afbbcf278234366d04f" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 37689 } }
I20260808 14:32:46.601054 25551 leader_election.cc:288] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20c8a943104740ae80d2d8a4afd40732 (127.7.125.66:34299), 909465f4961c4c42b3a2f8876628cd13 (127.7.125.67:34405)
W20260808 14:32:46.601912 25094 leader_election.cc:341] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20c8a943104740ae80d2d8a4afd40732 (127.7.125.66:34299): Not found: Tablet not found: c136be247ba943b8b2ca37edf4f96818
W20260808 14:32:46.602010 25096 leader_election.cc:341] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 909465f4961c4c42b3a2f8876628cd13 (127.7.125.67:34405): Not found: Tablet not found: c136be247ba943b8b2ca37edf4f96818
I20260808 14:32:46.602118 25096 leader_election.cc:302] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9894455cfa84afbbcf278234366d04f; no voters: 20c8a943104740ae80d2d8a4afd40732, 909465f4961c4c42b3a2f8876628cd13
I20260808 14:32:46.602308 25551 raft_consensus.cc:2726] T c136be247ba943b8b2ca37edf4f96818 P e9894455cfa84afbbcf278234366d04f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 14:32:46.629453 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:46.629038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:35021 (local address 127.7.125.66:42071)
0808 14:32:46.629285 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:46.629338 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.126:35021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":81,"threads_started":1}
I20260808 14:32:46.798519 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 25038
I20260808 14:32:46.806632 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 25215
I20260808 14:32:46.813725 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 25358
2026-08-08T14:32:46Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59501 ms)
[----------] 30 tests from SecurityITest (249914 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:32:46 dist-test-slave-w206 krb5kdc[25555](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:32:46 dist-test-slave-w206 krb5kdc[25555](info): set up 2 sockets
Aug 08 14:32:46 dist-test-slave-w206 krb5kdc[25555](info): commencing operation
krb5kdc: starting...
W20260808 14:32:48.004205 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:32:48 dist-test-slave-w206 krb5kdc[25555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:32:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:32:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:32:48.139322 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:40971
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:35861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:40971
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:32:48.258586 25569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:48.258905 25569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:48.258981 25569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:48.263096 25569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:32:48.263196 25569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:48.263234 25569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:48.263281 25569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:32:48.263316 25569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:32:48.268815 25569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:35861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:40971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:40971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:48.269995 25569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:48.271085 25569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:32:48.280182 25569 server_base.cc:1034] running on GCE node
W20260808 14:32:48.280213 25575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:48.280442 25577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:48.280213 25574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:48.281030 25569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:48.282253 25569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:48.283444 25569 hybrid_clock.cc:648] HybridClock initialized: now 1786199568283423 us; error 34 us; skew 500 ppm
Aug 08 14:32:48 dist-test-slave-w206 krb5kdc[25555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199568, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:32:48.286396 25569 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:32:48.287423 25569 webserver.cc:460] Webserver started at http://127.7.125.126:34415/ using document root <none> and password file <none>
I20260808 14:32:48.287981 25569 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:48.288038 25569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:48.288210 25569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:48.290277 25569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "4d27cfa516384736baa841b1ea0351e3"
format_stamp: "Formatted at 2026-08-08 14:32:48 on dist-test-slave-w206"
server_key: "96f59942ba21157033feca662313be7f"
server_key_iv: "8db1d1e5dae2551896c5a9ab83d57f9b"
server_key_version: "encryptionkey@0"
I20260808 14:32:48.290796 25569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "4d27cfa516384736baa841b1ea0351e3"
format_stamp: "Formatted at 2026-08-08 14:32:48 on dist-test-slave-w206"
server_key: "96f59942ba21157033feca662313be7f"
server_key_iv: "8db1d1e5dae2551896c5a9ab83d57f9b"
server_key_version: "encryptionkey@0"
I20260808 14:32:48.294644 25569 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:32:48.297197 25584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:48.298528 25569 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:32:48.298647 25569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "4d27cfa516384736baa841b1ea0351e3"
format_stamp: "Formatted at 2026-08-08 14:32:48 on dist-test-slave-w206"
server_key: "96f59942ba21157033feca662313be7f"
server_key_iv: "8db1d1e5dae2551896c5a9ab83d57f9b"
server_key_version: "encryptionkey@0"
I20260808 14:32:48.298763 25569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:48.311144 25569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:48.314810 25569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:48.315012 25569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:48.323122 25569 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:40971
I20260808 14:32:48.323148 25641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:40971 every 8 connection(s)
I20260808 14:32:48.324198 25569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:32:48.327104 25642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:48.327108 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 25569
I20260808 14:32:48.327257 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:32:48.327620 7669 external_mini_cluster.cc:1428] Setting key bcdfb368900b3f5a19d4e04c09399455
I20260808 14:32:48.334458 25642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3: Bootstrap starting.
I20260808 14:32:48.337035 25642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:48.337795 25642 log.cc:824] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:32:48 dist-test-slave-w206 krb5kdc[25555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:32:48.339888 25642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3: No bootstrap required, opened a new log
I20260808 14:32:48.343542 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:48.329218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58388 (local address 127.7.125.126:40971)
0808 14:32:48.329661 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:48.329681 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:48.329725 (+ 44us) server_negotiation.cc:407] Connection header received
0808 14:32:48.330496 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:48.330521 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:48.330953 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:48.331366 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:48.332171 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:48.333753 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:48.334863 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:48.335566 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:48.335752 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:48.338487 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:48.338493 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:48.338507 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:48.338536 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:48.340474 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:48.340964 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:48.340970 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:48.340974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:48.341089 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:48.341400 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:48.341404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:48.341406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:48.341819 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:48.342032 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:48.342365 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 14:32:48.342663 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":288,"thread_start_us":121,"threads_started":1}
I20260808 14:32:48.343734 25642 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d27cfa516384736baa841b1ea0351e3" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 40971 } }
I20260808 14:32:48.343988 25642 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:48.344064 25642 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d27cfa516384736baa841b1ea0351e3, State: Initialized, Role: FOLLOWER
I20260808 14:32:48.344517 25642 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d27cfa516384736baa841b1ea0351e3" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 40971 } }
I20260808 14:32:48.344668 25642 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:32:48.344765 25642 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:32:48.344900 25642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:48.345901 25642 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d27cfa516384736baa841b1ea0351e3" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 40971 } }
I20260808 14:32:48.346235 25642 leader_election.cc:302] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d27cfa516384736baa841b1ea0351e3; no voters:
I20260808 14:32:48.346515 25642 leader_election.cc:288] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:32:48.346676 25649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:32:48.346921 25649 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3 [term 1 LEADER]: Becoming Leader. State: Replica: 4d27cfa516384736baa841b1ea0351e3, State: Running, Role: LEADER
I20260808 14:32:48.347343 25649 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d27cfa516384736baa841b1ea0351e3" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 40971 } }
I20260808 14:32:48.347784 25642 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:32:48.348961 25650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d27cfa516384736baa841b1ea0351e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d27cfa516384736baa841b1ea0351e3" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 40971 } } }
I20260808 14:32:48.349191 25650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3 [sys.catalog]: This master's current role is: LEADER
I20260808 14:32:48.349274 25651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d27cfa516384736baa841b1ea0351e3. Latest consensus state: current_term: 1 leader_uuid: "4d27cfa516384736baa841b1ea0351e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d27cfa516384736baa841b1ea0351e3" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 40971 } } }
I20260808 14:32:48.349466 25651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3 [sys.catalog]: This master's current role is: LEADER
I20260808 14:32:48.349666 25658 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:32:48.352921 25658 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:32:48.358566 25658 catalog_manager.cc:1347] Generated new cluster ID: cbe650a74b5041bc871f7fd5058a3265
I20260808 14:32:48.358654 25658 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:32:48.382112 25658 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:32:48.383256 25658 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:32:48.392442 25658 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3: Generated new TSK 0
I20260808 14:32:48.393208 25658 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.125.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
I20260808 14:32:48.432029 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:40971
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:35861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:32:48.547587 25670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:48.547869 25670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:48.547963 25670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:48.552439 25670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:48.552502 25670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:48.552564 25670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:32:48.557817 25670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:35861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:40971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:48.558902 25670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:48.559942 25670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:48.569737 25675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:48.569767 25678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:48.569778 25676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:48.570014 25670 server_base.cc:1034] running on GCE node
I20260808 14:32:48.570523 25670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:48.571285 25670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:48.572490 25670 hybrid_clock.cc:648] HybridClock initialized: now 1786199568572474 us; error 36 us; skew 500 ppm
Aug 08 14:32:48 dist-test-slave-w206 krb5kdc[25555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199568, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:32:48.575695 25670 init.cc:377] Logged in from keytab as kudu/127.7.125.65@KRBTEST.COM (short username kudu)
I20260808 14:32:48.576570 25670 webserver.cc:460] Webserver started at http://127.7.125.65:35279/ using document root <none> and password file <none>
I20260808 14:32:48.577109 25670 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:48.577179 25670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:48.577353 25670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:48.579483 25670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "8dc54bdcf41f42ec9618ad14db595f35"
format_stamp: "Formatted at 2026-08-08 14:32:48 on dist-test-slave-w206"
server_key: "9289b419cce4432cd559a8a900d01b5d"
server_key_iv: "229c0931abcf1e25f5edaf090f081fc5"
server_key_version: "encryptionkey@0"
I20260808 14:32:48.580052 25670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "8dc54bdcf41f42ec9618ad14db595f35"
format_stamp: "Formatted at 2026-08-08 14:32:48 on dist-test-slave-w206"
server_key: "9289b419cce4432cd559a8a900d01b5d"
server_key_iv: "229c0931abcf1e25f5edaf090f081fc5"
server_key_version: "encryptionkey@0"
I20260808 14:32:48.584030 25670 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:32:48.586536 25685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:48.587941 25670 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:32:48.588078 25670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "8dc54bdcf41f42ec9618ad14db595f35"
format_stamp: "Formatted at 2026-08-08 14:32:48 on dist-test-slave-w206"
server_key: "9289b419cce4432cd559a8a900d01b5d"
server_key_iv: "229c0931abcf1e25f5edaf090f081fc5"
server_key_version: "encryptionkey@0"
I20260808 14:32:48.588227 25670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:48.620644 25670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:48.624534 25670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:48.624778 25670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:48.625478 25670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:48.626415 25670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:32:48.626468 25670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:48.626544 25670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:32:48.626580 25670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:48.636413 25670 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:40383
I20260808 14:32:48.636441 25806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:40383 every 8 connection(s)
I20260808 14:32:48.637411 25670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:32:48.640036 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 25670
I20260808 14:32:48.640187 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:32:48.640583 7669 external_mini_cluster.cc:1428] Setting key b8a39e33e6ce6906ff7382832afa3177
Aug 08 14:32:48 dist-test-slave-w206 krb5kdc[25555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199568, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.125.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.66@KRBTEST.COM" created.
I20260808 14:32:48.654506 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:48.639475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:52077 (local address 127.7.125.126:40971)
0808 14:32:48.639633 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:48.639637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:48.640673 (+ 1036us) server_negotiation.cc:407] Connection header received
0808 14:32:48.642371 (+ 1698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:48.642374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:48.642431 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:48.642528 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:48.643801 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:48.644768 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:48.646552 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:48.646806 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:48.646894 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:48.650303 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:48.650308 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:48.650311 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:48.650347 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:48.652135 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:48.652692 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:48.652696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:48.652698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:48.652823 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:48.653196 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:48.653209 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:48.653211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:48.653414 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:48.653533 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:48.654133 (+ 600us) server_negotiation.cc:299] Negotiation successful
0808 14:32:48.654282 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 14:32:48.655673 25810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:48.639957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:40971 (local address 127.7.125.65:52077)
0808 14:32:48.640522 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:48.640566 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:48.642123 (+ 1557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:48.642701 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:48.642708 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:48.643159 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:48.643648 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:48.643662 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:48.644934 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:48.644941 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:48.646416 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:48.646432 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:48.646996 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:48.647002 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:48.647248 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:48.648054 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:32:48.648082 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:48.650117 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:48.652270 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:48.652276 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:48.652293 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:48.652576 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:48.652927 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:48.652931 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:48.652934 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:48.653091 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:48.653532 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:48.653539 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:48.653815 (+ 276us) client_negotiation.cc:769] Sending connection context
0808 14:32:48.654074 (+ 259us) client_negotiation.cc:240] Negotiation successful
0808 14:32:48.654429 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":168,"threads_started":1}
I20260808 14:32:48.657061 25808 heartbeater.cc:347] Connected to a master server at 127.7.125.126:40971
I20260808 14:32:48.657336 25808 heartbeater.cc:464] Registering TS with master...
I20260808 14:32:48.657876 25808 heartbeater.cc:511] Master 127.7.125.126:40971 requested a full tablet report, sending...
I20260808 14:32:48.659677 25604 ts_manager.cc:194] Registered new tserver with Master: 8dc54bdcf41f42ec9618ad14db595f35 (127.7.125.65:40383)
I20260808 14:32:48.661449 25604 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:52077
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
I20260808 14:32:48.672289 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:40971
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:35861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:32:48.789651 25813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:48.789932 25813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:48.789980 25813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:48.794732 25813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:48.794802 25813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:48.794869 25813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:32:48.800518 25813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:35861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:40971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:48.801579 25813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:48.802656 25813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:48.812439 25819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:48.812717 25813 server_base.cc:1034] running on GCE node
W20260808 14:32:48.812471 25818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:48.812463 25821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:48.813256 25813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:48.813933 25813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:48.815109 25813 hybrid_clock.cc:648] HybridClock initialized: now 1786199568815075 us; error 50 us; skew 500 ppm
Aug 08 14:32:48 dist-test-slave-w206 krb5kdc[25555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199568, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:32:48.822729 25813 init.cc:377] Logged in from keytab as kudu/127.7.125.66@KRBTEST.COM (short username kudu)
I20260808 14:32:48.823860 25813 webserver.cc:460] Webserver started at http://127.7.125.66:33617/ using document root <none> and password file <none>
I20260808 14:32:48.824471 25813 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:48.824568 25813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:48.824815 25813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:48.826964 25813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "4f78ca580d054134a91cd7381c1ac124"
format_stamp: "Formatted at 2026-08-08 14:32:48 on dist-test-slave-w206"
server_key: "d3f4c519a711445866f06cf7875255c4"
server_key_iv: "31c20e4bcd2edf0698ee967c27baf55c"
server_key_version: "encryptionkey@0"
I20260808 14:32:48.827621 25813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "4f78ca580d054134a91cd7381c1ac124"
format_stamp: "Formatted at 2026-08-08 14:32:48 on dist-test-slave-w206"
server_key: "d3f4c519a711445866f06cf7875255c4"
server_key_iv: "31c20e4bcd2edf0698ee967c27baf55c"
server_key_version: "encryptionkey@0"
I20260808 14:32:48.831468 25813 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 14:32:48.834060 25828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:48.835428 25813 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:32:48.835574 25813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "4f78ca580d054134a91cd7381c1ac124"
format_stamp: "Formatted at 2026-08-08 14:32:48 on dist-test-slave-w206"
server_key: "d3f4c519a711445866f06cf7875255c4"
server_key_iv: "31c20e4bcd2edf0698ee967c27baf55c"
server_key_version: "encryptionkey@0"
I20260808 14:32:48.835703 25813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:48.851866 25813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:48.855324 25813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:48.855547 25813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:48.856156 25813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:48.857010 25813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:32:48.857084 25813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:48.857156 25813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:32:48.857206 25813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:48.866470 25813 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:38209
I20260808 14:32:48.866552 25948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:38209 every 8 connection(s)
I20260808 14:32:48.867684 25813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:32:48.870165 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 25813
I20260808 14:32:48.870292 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:32:48.870625 7669 external_mini_cluster.cc:1428] Setting key f9deef338d3b6e724cda46ddad787fee
Aug 08 14:32:48 dist-test-slave-w206 krb5kdc[25555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199568, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.125.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.67@KRBTEST.COM" created.
I20260808 14:32:48.884222 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:48.871118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:44937 (local address 127.7.125.126:40971)
0808 14:32:48.871347 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:48.871352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:48.871370 (+ 18us) server_negotiation.cc:407] Connection header received
0808 14:32:48.873090 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:48.873093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:48.873136 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:48.873198 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:48.874934 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:48.875528 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:48.876758 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:48.877003 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:48.877129 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:48.879913 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:48.879930 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:48.879932 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:48.879958 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:48.881660 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:48.882286 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:48.882290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:48.882292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:48.882387 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:48.882811 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:48.882817 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:48.882818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:48.883112 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:48.883243 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:48.883876 (+ 633us) server_negotiation.cc:299] Negotiation successful
0808 14:32:48.884021 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260808 14:32:48.885358 25951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:48.870473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:40971 (local address 127.7.125.66:44937)
0808 14:32:48.871157 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:48.871224 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:48.872832 (+ 1608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:48.873400 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:48.873413 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:48.873997 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:48.874761 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:48.874780 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:48.875690 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:48.875694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:48.876603 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:48.876614 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:48.877176 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:48.877181 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:48.877377 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:48.878061 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:32:48.878080 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:48.879693 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:48.881791 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:48.881797 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:48.881814 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:48.882105 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:48.882501 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:48.882504 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:48.882506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:48.882694 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:48.883239 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:48.883266 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:48.883607 (+ 341us) client_negotiation.cc:769] Sending connection context
0808 14:32:48.883832 (+ 225us) client_negotiation.cc:240] Negotiation successful
0808 14:32:48.884217 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":161,"threads_started":1}
I20260808 14:32:48.886747 25949 heartbeater.cc:347] Connected to a master server at 127.7.125.126:40971
I20260808 14:32:48.887094 25949 heartbeater.cc:464] Registering TS with master...
I20260808 14:32:48.887693 25949 heartbeater.cc:511] Master 127.7.125.126:40971 requested a full tablet report, sending...
I20260808 14:32:48.888895 25604 ts_manager.cc:194] Registered new tserver with Master: 4f78ca580d054134a91cd7381c1ac124 (127.7.125.66:38209)
I20260808 14:32:48.889573 25604 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:44937
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
I20260808 14:32:48.902485 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:40971
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:35861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:32:49.014802 25954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:49.015187 25954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:49.015286 25954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:49.019461 25954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:49.019577 25954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:49.019678 25954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:32:49.025332 25954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:35861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:40971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:49.026459 25954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:49.027590 25954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:49.037521 25961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:49.037524 25964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:49.037492 25960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:49.037672 25954 server_base.cc:1034] running on GCE node
I20260808 14:32:49.038197 25954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:49.038945 25954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:49.040161 25954 hybrid_clock.cc:648] HybridClock initialized: now 1786199569040130 us; error 53 us; skew 500 ppm
Aug 08 14:32:49 dist-test-slave-w206 krb5kdc[25555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199569, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:32:49.043097 25954 init.cc:377] Logged in from keytab as kudu/127.7.125.67@KRBTEST.COM (short username kudu)
I20260808 14:32:49.043979 25954 webserver.cc:460] Webserver started at http://127.7.125.67:45553/ using document root <none> and password file <none>
I20260808 14:32:49.044551 25954 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:49.044613 25954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:49.044824 25954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:49.046844 25954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "7edbfc09954d4b4ab8e4d3df152db05e"
format_stamp: "Formatted at 2026-08-08 14:32:49 on dist-test-slave-w206"
server_key: "c734377973e51713ed3087a51a07f7a1"
server_key_iv: "68c80f072b2219886038fdf6563cd521"
server_key_version: "encryptionkey@0"
I20260808 14:32:49.047545 25954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "7edbfc09954d4b4ab8e4d3df152db05e"
format_stamp: "Formatted at 2026-08-08 14:32:49 on dist-test-slave-w206"
server_key: "c734377973e51713ed3087a51a07f7a1"
server_key_iv: "68c80f072b2219886038fdf6563cd521"
server_key_version: "encryptionkey@0"
I20260808 14:32:49.051466 25954 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:32:49.053911 25972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:49.055258 25954 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:32:49.055389 25954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "7edbfc09954d4b4ab8e4d3df152db05e"
format_stamp: "Formatted at 2026-08-08 14:32:49 on dist-test-slave-w206"
server_key: "c734377973e51713ed3087a51a07f7a1"
server_key_iv: "68c80f072b2219886038fdf6563cd521"
server_key_version: "encryptionkey@0"
I20260808 14:32:49.055502 25954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:49.069891 25954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:49.073180 25954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:49.073406 25954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:49.074018 25954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:49.074887 25954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:32:49.074961 25954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:49.075107 25954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:32:49.075157 25954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:49.084841 25954 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:43951
I20260808 14:32:49.084874 26088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:43951 every 8 connection(s)
I20260808 14:32:49.085809 25954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:32:49.089780 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 25954
I20260808 14:32:49.089939 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:32:49.090298 7669 external_mini_cluster.cc:1428] Setting key ed1e1d5359cf3d39c71aad8f302ddd8b
Aug 08 14:32:49 dist-test-slave-w206 krb5kdc[25555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199569, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:32:49.100634 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:49.087895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:47751 (local address 127.7.125.126:40971)
0808 14:32:49.088050 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:49.088054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:49.088986 (+ 932us) server_negotiation.cc:407] Connection header received
0808 14:32:49.090332 (+ 1346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:49.090338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:49.090407 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:49.090517 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:49.092043 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.092863 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.093987 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.094223 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.094323 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:49.096885 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:49.096889 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:49.096892 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:49.096918 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:49.098625 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:49.099205 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:49.099209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:49.099211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:49.099304 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:49.099619 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:49.099627 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:49.099629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:49.099836 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:49.099931 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:49.100396 (+ 465us) server_negotiation.cc:299] Negotiation successful
0808 14:32:49.100505 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 14:32:49.101477 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:49.088325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:40971 (local address 127.7.125.67:47751)
0808 14:32:49.088813 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:49.088857 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:49.090109 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:49.090680 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:49.090688 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:49.091258 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:49.091903 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:49.091922 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.092992 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.092995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:49.093871 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:49.093880 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.094369 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.094372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:49.094512 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:49.095172 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:32:49.095188 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:49.096725 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:49.098737 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:49.098742 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:49.098754 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:49.099093 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:49.099397 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:49.099399 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:49.099401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:49.099528 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:49.099938 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:49.099943 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:49.100173 (+ 230us) client_negotiation.cc:769] Sending connection context
0808 14:32:49.100320 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 14:32:49.100575 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":115,"threads_started":1}
I20260808 14:32:49.102890 26090 heartbeater.cc:347] Connected to a master server at 127.7.125.126:40971
I20260808 14:32:49.103163 26090 heartbeater.cc:464] Registering TS with master...
I20260808 14:32:49.103772 26090 heartbeater.cc:511] Master 127.7.125.126:40971 requested a full tablet report, sending...
I20260808 14:32:49.104840 25604 ts_manager.cc:194] Registered new tserver with Master: 7edbfc09954d4b4ab8e4d3df152db05e (127.7.125.67:43951)
I20260808 14:32:49.105387 25604 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:47751
I20260808 14:32:49.116909 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:32:49.127034 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:49.119041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58392 (local address 127.7.125.126:40971)
0808 14:32:49.119181 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:49.119186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:49.119374 (+ 188us) server_negotiation.cc:407] Connection header received
0808 14:32:49.119562 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:49.119566 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:49.119620 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:49.119706 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:49.120472 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.121009 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.121871 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.122108 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.122220 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:49.123719 (+ 1499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:49.123723 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:49.123726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:49.123753 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:49.125231 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:49.125667 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:49.125671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:49.125673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:49.125755 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:49.126050 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:49.126054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:49.126056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:49.126294 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:49.126436 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:49.126691 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 14:32:49.126811 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 14:32:49.129978 25604 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:32:49.133080 25604 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:32:49.147883 26101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:49.142193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:40383 (local address 127.0.0.1:47102)
0808 14:32:49.142897 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:49.142954 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:49.143105 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:49.143639 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:49.143645 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:49.143665 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:49.143809 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:49.143818 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.145338 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.145341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:49.146394 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:49.146405 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.147339 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.147342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:49.147425 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:49.147455 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 14:32:49.147586 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 14:32:49.147708 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"mutex_wait_us":58,"thread_start_us":127,"threads_started":1}
I20260808 14:32:49.147877 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:49.142730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47102 (local address 127.7.125.65:40383)
0808 14:32:49.143055 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:49.143072 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:49.143098 (+ 26us) server_negotiation.cc:407] Connection header received
0808 14:32:49.143260 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:49.143267 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:49.143492 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:49.143660 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:49.143953 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.145186 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.146521 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.147218 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.147331 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:49.147489 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:49.147624 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 14:32:49.147722 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":123,"threads_started":1}
I20260808 14:32:49.148622 26104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:49.141931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:38209 (local address 127.0.0.1:40714)
0808 14:32:49.143359 (+ 1428us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:49.143387 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:49.143463 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:49.143966 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:49.143969 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:49.143984 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:49.144082 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:49.144092 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.145318 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.145321 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:49.146926 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:49.146932 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.148206 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.148211 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:49.148303 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:49.148327 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 14:32:49.148401 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 14:32:49.148478 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"mutex_wait_us":163,"thread_start_us":84,"threads_started":1}
I20260808 14:32:49.149237 26106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:49.142763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40714 (local address 127.7.125.66:38209)
0808 14:32:49.143277 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:49.143291 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:49.143550 (+ 259us) server_negotiation.cc:407] Connection header received
0808 14:32:49.143627 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:49.143632 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:49.143804 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:49.143955 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:49.144211 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.145176 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.147086 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.147945 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.148063 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:49.148916 (+ 853us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:49.148971 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 14:32:49.149062 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":151,"threads_started":1}
I20260808 14:32:49.151523 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:49.142326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:43951 (local address 127.0.0.1:39502)
0808 14:32:49.143359 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:49.143373 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:49.143455 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:49.144128 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:49.144134 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:49.144155 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:49.144230 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:49.144235 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.145549 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.145552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:49.146919 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:49.146931 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.151082 (+ 4151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.151086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:49.151182 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:49.151208 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 14:32:49.151295 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 14:32:49.151374 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":972,"mutex_wait_us":92,"spinlock_wait_cycles":384,"thread_start_us":94,"threads_started":1}
I20260808 14:32:49.151959 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:49.143133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39502 (local address 127.7.125.67:43951)
0808 14:32:49.143601 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:49.143619 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:49.143641 (+ 22us) server_negotiation.cc:407] Connection header received
0808 14:32:49.143713 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:49.143718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:49.143925 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:49.144079 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:49.144345 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.145417 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.147564 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.149966 (+ 2402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.150098 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:49.150238 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:49.151348 (+ 1110us) server_negotiation.cc:299] Negotiation successful
0808 14:32:49.151467 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":158,"threads_started":1}
I20260808 14:32:49.151469 25735 tablet_service.cc:1467] Processing CreateTablet for tablet 98cead73889c4e7a858b1b860862e6a3 (DEFAULT_TABLE table=test-table [id=77abf3bbd7f44ec5b72844d81b76f2bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:32:49.152146 25878 tablet_service.cc:1467] Processing CreateTablet for tablet 98cead73889c4e7a858b1b860862e6a3 (DEFAULT_TABLE table=test-table [id=77abf3bbd7f44ec5b72844d81b76f2bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:32:49.152691 25735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98cead73889c4e7a858b1b860862e6a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:49.153209 25878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98cead73889c4e7a858b1b860862e6a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:49.154829 26021 tablet_service.cc:1467] Processing CreateTablet for tablet 98cead73889c4e7a858b1b860862e6a3 (DEFAULT_TABLE table=test-table [id=77abf3bbd7f44ec5b72844d81b76f2bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:32:49.155862 26021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98cead73889c4e7a858b1b860862e6a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:49.159406 26108 tablet_bootstrap.cc:492] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35: Bootstrap starting.
I20260808 14:32:49.160934 26109 tablet_bootstrap.cc:492] T 98cead73889c4e7a858b1b860862e6a3 P 4f78ca580d054134a91cd7381c1ac124: Bootstrap starting.
I20260808 14:32:49.162127 26108 tablet_bootstrap.cc:654] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:49.163198 26109 tablet_bootstrap.cc:654] T 98cead73889c4e7a858b1b860862e6a3 P 4f78ca580d054134a91cd7381c1ac124: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:49.163328 26108 log.cc:824] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:49.164047 26110 tablet_bootstrap.cc:492] T 98cead73889c4e7a858b1b860862e6a3 P 7edbfc09954d4b4ab8e4d3df152db05e: Bootstrap starting.
I20260808 14:32:49.164465 26109 log.cc:824] T 98cead73889c4e7a858b1b860862e6a3 P 4f78ca580d054134a91cd7381c1ac124: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:49.165707 26108 tablet_bootstrap.cc:492] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35: No bootstrap required, opened a new log
I20260808 14:32:49.165952 26108 ts_tablet_manager.cc:1397] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 14:32:49.166281 26109 tablet_bootstrap.cc:492] T 98cead73889c4e7a858b1b860862e6a3 P 4f78ca580d054134a91cd7381c1ac124: No bootstrap required, opened a new log
I20260808 14:32:49.166324 26110 tablet_bootstrap.cc:654] T 98cead73889c4e7a858b1b860862e6a3 P 7edbfc09954d4b4ab8e4d3df152db05e: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:49.166492 26109 ts_tablet_manager.cc:1397] T 98cead73889c4e7a858b1b860862e6a3 P 4f78ca580d054134a91cd7381c1ac124: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:32:49.167670 26110 log.cc:824] T 98cead73889c4e7a858b1b860862e6a3 P 7edbfc09954d4b4ab8e4d3df152db05e: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:49.169975 26110 tablet_bootstrap.cc:492] T 98cead73889c4e7a858b1b860862e6a3 P 7edbfc09954d4b4ab8e4d3df152db05e: No bootstrap required, opened a new log
I20260808 14:32:49.170186 26110 ts_tablet_manager.cc:1397] T 98cead73889c4e7a858b1b860862e6a3 P 7edbfc09954d4b4ab8e4d3df152db05e: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 14:32:49.169972 26109 raft_consensus.cc:348] T 98cead73889c4e7a858b1b860862e6a3 P 4f78ca580d054134a91cd7381c1ac124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f78ca580d054134a91cd7381c1ac124" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38209 } } peers { permanent_uuid: "8dc54bdcf41f42ec9618ad14db595f35" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 40383 } } peers { permanent_uuid: "7edbfc09954d4b4ab8e4d3df152db05e" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 43951 } }
I20260808 14:32:49.170293 26109 raft_consensus.cc:374] T 98cead73889c4e7a858b1b860862e6a3 P 4f78ca580d054134a91cd7381c1ac124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:49.170349 26109 raft_consensus.cc:729] T 98cead73889c4e7a858b1b860862e6a3 P 4f78ca580d054134a91cd7381c1ac124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f78ca580d054134a91cd7381c1ac124, State: Initialized, Role: FOLLOWER
I20260808 14:32:49.170571 26108 raft_consensus.cc:348] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f78ca580d054134a91cd7381c1ac124" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38209 } } peers { permanent_uuid: "8dc54bdcf41f42ec9618ad14db595f35" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 40383 } } peers { permanent_uuid: "7edbfc09954d4b4ab8e4d3df152db05e" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 43951 } }
I20260808 14:32:49.170830 26108 raft_consensus.cc:374] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:49.170885 26108 raft_consensus.cc:729] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dc54bdcf41f42ec9618ad14db595f35, State: Initialized, Role: FOLLOWER
I20260808 14:32:49.170854 26109 consensus_queue.cc:260] T 98cead73889c4e7a858b1b860862e6a3 P 4f78ca580d054134a91cd7381c1ac124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f78ca580d054134a91cd7381c1ac124" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38209 } } peers { permanent_uuid: "8dc54bdcf41f42ec9618ad14db595f35" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 40383 } } peers { permanent_uuid: "7edbfc09954d4b4ab8e4d3df152db05e" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 43951 } }
I20260808 14:32:49.171416 26108 consensus_queue.cc:260] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f78ca580d054134a91cd7381c1ac124" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38209 } } peers { permanent_uuid: "8dc54bdcf41f42ec9618ad14db595f35" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 40383 } } peers { permanent_uuid: "7edbfc09954d4b4ab8e4d3df152db05e" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 43951 } }
I20260808 14:32:49.172072 26109 ts_tablet_manager.cc:1428] T 98cead73889c4e7a858b1b860862e6a3 P 4f78ca580d054134a91cd7381c1ac124: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 14:32:49.172163 25808 heartbeater.cc:503] Master 127.7.125.126:40971 was elected leader, sending a full tablet report...
I20260808 14:32:49.172398 25949 heartbeater.cc:503] Master 127.7.125.126:40971 was elected leader, sending a full tablet report...
I20260808 14:32:49.172397 26108 ts_tablet_manager.cc:1428] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 14:32:49.174045 26110 raft_consensus.cc:348] T 98cead73889c4e7a858b1b860862e6a3 P 7edbfc09954d4b4ab8e4d3df152db05e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f78ca580d054134a91cd7381c1ac124" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38209 } } peers { permanent_uuid: "8dc54bdcf41f42ec9618ad14db595f35" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 40383 } } peers { permanent_uuid: "7edbfc09954d4b4ab8e4d3df152db05e" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 43951 } }
I20260808 14:32:49.174333 26110 raft_consensus.cc:374] T 98cead73889c4e7a858b1b860862e6a3 P 7edbfc09954d4b4ab8e4d3df152db05e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:49.174400 26110 raft_consensus.cc:729] T 98cead73889c4e7a858b1b860862e6a3 P 7edbfc09954d4b4ab8e4d3df152db05e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7edbfc09954d4b4ab8e4d3df152db05e, State: Initialized, Role: FOLLOWER
I20260808 14:32:49.174865 26110 consensus_queue.cc:260] T 98cead73889c4e7a858b1b860862e6a3 P 7edbfc09954d4b4ab8e4d3df152db05e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f78ca580d054134a91cd7381c1ac124" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38209 } } peers { permanent_uuid: "8dc54bdcf41f42ec9618ad14db595f35" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 40383 } } peers { permanent_uuid: "7edbfc09954d4b4ab8e4d3df152db05e" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 43951 } }
I20260808 14:32:49.175614 26090 heartbeater.cc:503] Master 127.7.125.126:40971 was elected leader, sending a full tablet report...
I20260808 14:32:49.175901 26110 ts_tablet_manager.cc:1428] T 98cead73889c4e7a858b1b860862e6a3 P 7edbfc09954d4b4ab8e4d3df152db05e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260808 14:32:49.336790 26091 tablet.cc:2369] T 98cead73889c4e7a858b1b860862e6a3 P 7edbfc09954d4b4ab8e4d3df152db05e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:32:49.370005 25950 tablet.cc:2369] T 98cead73889c4e7a858b1b860862e6a3 P 4f78ca580d054134a91cd7381c1ac124: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:32:49.388922 25809 tablet.cc:2369] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:32:49.463867 26117 raft_consensus.cc:482] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:32:49.464084 26117 raft_consensus.cc:504] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f78ca580d054134a91cd7381c1ac124" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38209 } } peers { permanent_uuid: "8dc54bdcf41f42ec9618ad14db595f35" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 40383 } } peers { permanent_uuid: "7edbfc09954d4b4ab8e4d3df152db05e" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 43951 } }
I20260808 14:32:49.465261 26117 leader_election.cc:288] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f78ca580d054134a91cd7381c1ac124 (127.7.125.66:38209), 7edbfc09954d4b4ab8e4d3df152db05e (127.7.125.67:43951)
I20260808 14:32:49.469422 26106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:49.465616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:38585 (local address 127.7.125.66:38209)
0808 14:32:49.465767 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:49.465771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:49.466051 (+ 280us) server_negotiation.cc:407] Connection header received
0808 14:32:49.466281 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:49.466284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:49.466331 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:49.466426 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:49.466753 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.467319 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.468426 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.468822 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.468949 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:49.469004 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:49.469215 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 14:32:49.469293 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 14:32:49.469448 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:49.465731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:48087 (local address 127.7.125.67:43951)
0808 14:32:49.465927 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:49.465933 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:49.466166 (+ 233us) server_negotiation.cc:407] Connection header received
0808 14:32:49.466329 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:49.466332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:49.466386 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:49.466476 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:49.466788 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.467342 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.468448 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.468849 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.468984 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:49.469038 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:49.469232 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 14:32:49.469311 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 14:32:49.469476 26119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:49.465595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:43951 (local address 127.7.125.65:48087)
0808 14:32:49.466062 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:49.466074 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:49.466204 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:49.466489 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:49.466492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:49.466531 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:49.466664 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:49.466670 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.467466 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.467471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:49.468316 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:49.468325 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.468976 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.468980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:49.469109 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:49.469126 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 14:32:49.469214 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 14:32:49.469311 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":46,"spinlock_wait_cycles":1408,"thread_start_us":118,"threads_started":1}
I20260808 14:32:49.469455 26120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:49.465511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:38209 (local address 127.7.125.65:38585)
0808 14:32:49.465934 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:49.465949 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:49.466100 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:49.466438 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:49.466441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:49.466457 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:49.466634 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:49.466640 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.467435 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.467439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:49.468296 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:49.468305 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.468944 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.468948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:49.469046 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:49.469100 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 14:32:49.469189 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 14:32:49.469279 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":31,"spinlock_wait_cycles":33792,"thread_start_us":135,"threads_started":1}
I20260808 14:32:49.470196 26042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98cead73889c4e7a858b1b860862e6a3" candidate_uuid: "8dc54bdcf41f42ec9618ad14db595f35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7edbfc09954d4b4ab8e4d3df152db05e" is_pre_election: true
I20260808 14:32:49.470291 25899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98cead73889c4e7a858b1b860862e6a3" candidate_uuid: "8dc54bdcf41f42ec9618ad14db595f35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f78ca580d054134a91cd7381c1ac124" is_pre_election: true
I20260808 14:32:49.470530 26042 raft_consensus.cc:2454] T 98cead73889c4e7a858b1b860862e6a3 P 7edbfc09954d4b4ab8e4d3df152db05e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dc54bdcf41f42ec9618ad14db595f35 in term 0.
I20260808 14:32:49.470634 25899 raft_consensus.cc:2454] T 98cead73889c4e7a858b1b860862e6a3 P 4f78ca580d054134a91cd7381c1ac124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dc54bdcf41f42ec9618ad14db595f35 in term 0.
I20260808 14:32:49.471143 25688 leader_election.cc:302] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4f78ca580d054134a91cd7381c1ac124, 7edbfc09954d4b4ab8e4d3df152db05e, 8dc54bdcf41f42ec9618ad14db595f35; no voters:
I20260808 14:32:49.471386 26117 raft_consensus.cc:2781] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:32:49.471495 26117 raft_consensus.cc:482] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:32:49.471554 26117 raft_consensus.cc:3037] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:49.472615 26117 raft_consensus.cc:504] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f78ca580d054134a91cd7381c1ac124" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38209 } } peers { permanent_uuid: "8dc54bdcf41f42ec9618ad14db595f35" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 40383 } } peers { permanent_uuid: "7edbfc09954d4b4ab8e4d3df152db05e" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 43951 } }
I20260808 14:32:49.473022 26117 leader_election.cc:288] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35 [CANDIDATE]: Term 1 election: Requested vote from peers 4f78ca580d054134a91cd7381c1ac124 (127.7.125.66:38209), 7edbfc09954d4b4ab8e4d3df152db05e (127.7.125.67:43951)
I20260808 14:32:49.473387 25899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98cead73889c4e7a858b1b860862e6a3" candidate_uuid: "8dc54bdcf41f42ec9618ad14db595f35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f78ca580d054134a91cd7381c1ac124"
I20260808 14:32:49.473476 26042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98cead73889c4e7a858b1b860862e6a3" candidate_uuid: "8dc54bdcf41f42ec9618ad14db595f35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7edbfc09954d4b4ab8e4d3df152db05e"
I20260808 14:32:49.473536 25899 raft_consensus.cc:3037] T 98cead73889c4e7a858b1b860862e6a3 P 4f78ca580d054134a91cd7381c1ac124 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:49.473651 26042 raft_consensus.cc:3037] T 98cead73889c4e7a858b1b860862e6a3 P 7edbfc09954d4b4ab8e4d3df152db05e [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:49.474586 25899 raft_consensus.cc:2454] T 98cead73889c4e7a858b1b860862e6a3 P 4f78ca580d054134a91cd7381c1ac124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dc54bdcf41f42ec9618ad14db595f35 in term 1.
I20260808 14:32:49.474772 26042 raft_consensus.cc:2454] T 98cead73889c4e7a858b1b860862e6a3 P 7edbfc09954d4b4ab8e4d3df152db05e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dc54bdcf41f42ec9618ad14db595f35 in term 1.
I20260808 14:32:49.474926 25687 leader_election.cc:302] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f78ca580d054134a91cd7381c1ac124, 8dc54bdcf41f42ec9618ad14db595f35; no voters:
I20260808 14:32:49.475134 26117 raft_consensus.cc:2781] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:32:49.475389 26117 raft_consensus.cc:686] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35 [term 1 LEADER]: Becoming Leader. State: Replica: 8dc54bdcf41f42ec9618ad14db595f35, State: Running, Role: LEADER
I20260808 14:32:49.475719 26117 consensus_queue.cc:237] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f78ca580d054134a91cd7381c1ac124" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38209 } } peers { permanent_uuid: "8dc54bdcf41f42ec9618ad14db595f35" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 40383 } } peers { permanent_uuid: "7edbfc09954d4b4ab8e4d3df152db05e" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 43951 } }
I20260808 14:32:49.479162 25602 catalog_manager.cc:5495] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8dc54bdcf41f42ec9618ad14db595f35 (127.7.125.65). New cstate: current_term: 1 leader_uuid: "8dc54bdcf41f42ec9618ad14db595f35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f78ca580d054134a91cd7381c1ac124" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8dc54bdcf41f42ec9618ad14db595f35" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 40383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7edbfc09954d4b4ab8e4d3df152db05e" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 43951 } health_report { overall_health: UNKNOWN } } }
I20260808 14:32:49.511649 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:49.508345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47110 (local address 127.7.125.65:40383)
0808 14:32:49.508479 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:49.508483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:49.508496 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:32:49.508630 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:49.508633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:49.508691 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:49.508784 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:32:49.509017 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.509501 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.510333 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.510537 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.510610 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:49.510900 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:32:49.511333 (+ 433us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:32:49.511422 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:49.511490 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 14:32:49.511537 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 14:32:49.517800 26042 raft_consensus.cc:1261] T 98cead73889c4e7a858b1b860862e6a3 P 7edbfc09954d4b4ab8e4d3df152db05e [term 1 FOLLOWER]: Refusing update from remote peer 8dc54bdcf41f42ec9618ad14db595f35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:32:49.517851 25899 raft_consensus.cc:1261] T 98cead73889c4e7a858b1b860862e6a3 P 4f78ca580d054134a91cd7381c1ac124 [term 1 FOLLOWER]: Refusing update from remote peer 8dc54bdcf41f42ec9618ad14db595f35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:32:49.518641 26117 consensus_queue.cc:1035] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7edbfc09954d4b4ab8e4d3df152db05e" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 43951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:32:49.518878 26121 consensus_queue.cc:1035] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f78ca580d054134a91cd7381c1ac124" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:32:49.526654 26124 mvcc.cc:205] Tried to move back new op lower bound from 7316273436737015808 to 7316273436575256576. Current Snapshot: MvccSnapshot[applied={T|T < 7316273436737015808}]
I20260808 14:32:49.527041 26126 mvcc.cc:205] Tried to move back new op lower bound from 7316273436737015808 to 7316273436575256576. Current Snapshot: MvccSnapshot[applied={T|T < 7316273436737015808}]
I20260808 14:32:49.527432 26125 mvcc.cc:205] Tried to move back new op lower bound from 7316273436737015808 to 7316273436575256576. Current Snapshot: MvccSnapshot[applied={T|T < 7316273436737015808}]
I20260808 14:32:49.536617 25603 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:32:49.536809 25603 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:32:49.539350 25603 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d27cfa516384736baa841b1ea0351e3: Sending DeleteTablet for 3 replicas of tablet 98cead73889c4e7a858b1b860862e6a3
I20260808 14:32:49.540208 25735 tablet_service.cc:1514] Processing DeleteTablet for tablet 98cead73889c4e7a858b1b860862e6a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:32:49 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:47102
I20260808 14:32:49.540180 26021 tablet_service.cc:1514] Processing DeleteTablet for tablet 98cead73889c4e7a858b1b860862e6a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:32:49 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:39502
I20260808 14:32:49.540190 25878 tablet_service.cc:1514] Processing DeleteTablet for tablet 98cead73889c4e7a858b1b860862e6a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:32:49 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:40714
I20260808 14:32:49.540714 26134 tablet_replica.cc:330] T 98cead73889c4e7a858b1b860862e6a3 P 7edbfc09954d4b4ab8e4d3df152db05e: stopping tablet replica
I20260808 14:32:49.540933 26135 tablet_replica.cc:330] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35: stopping tablet replica
I20260808 14:32:49.541188 26134 raft_consensus.cc:2229] T 98cead73889c4e7a858b1b860862e6a3 P 7edbfc09954d4b4ab8e4d3df152db05e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:32:49.541188 26135 raft_consensus.cc:2229] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:32:49.541575 26135 raft_consensus.cc:2258] T 98cead73889c4e7a858b1b860862e6a3 P 8dc54bdcf41f42ec9618ad14db595f35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:32:49.541580 26134 raft_consensus.cc:2258] T 98cead73889c4e7a858b1b860862e6a3 P 7edbfc09954d4b4ab8e4d3df152db05e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:32:49.541724 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 25670
I20260808 14:32:49.542338 26136 tablet_replica.cc:330] T 98cead73889c4e7a858b1b860862e6a3 P 4f78ca580d054134a91cd7381c1ac124: stopping tablet replica
I20260808 14:32:49.542629 26136 raft_consensus.cc:2229] T 98cead73889c4e7a858b1b860862e6a3 P 4f78ca580d054134a91cd7381c1ac124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:32:49.542933 26136 raft_consensus.cc:2258] T 98cead73889c4e7a858b1b860862e6a3 P 4f78ca580d054134a91cd7381c1ac124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:32:49.544616 26134 ts_tablet_manager.cc:1905] T 98cead73889c4e7a858b1b860862e6a3 P 7edbfc09954d4b4ab8e4d3df152db05e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:32:49.545063 26136 ts_tablet_manager.cc:1905] T 98cead73889c4e7a858b1b860862e6a3 P 4f78ca580d054134a91cd7381c1ac124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:32:49.547224 26136 ts_tablet_manager.cc:1918] T 98cead73889c4e7a858b1b860862e6a3 P 4f78ca580d054134a91cd7381c1ac124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:32:49.547308 26136 log.cc:1196] T 98cead73889c4e7a858b1b860862e6a3 P 4f78ca580d054134a91cd7381c1ac124: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/98cead73889c4e7a858b1b860862e6a3
I20260808 14:32:49.547640 26136 ts_tablet_manager.cc:1939] T 98cead73889c4e7a858b1b860862e6a3 P 4f78ca580d054134a91cd7381c1ac124: Deleting consensus metadata
I20260808 14:32:49.548424 26134 ts_tablet_manager.cc:1918] T 98cead73889c4e7a858b1b860862e6a3 P 7edbfc09954d4b4ab8e4d3df152db05e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:32:49.548529 26134 log.cc:1196] T 98cead73889c4e7a858b1b860862e6a3 P 7edbfc09954d4b4ab8e4d3df152db05e: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/98cead73889c4e7a858b1b860862e6a3
I20260808 14:32:49.548940 26134 ts_tablet_manager.cc:1939] T 98cead73889c4e7a858b1b860862e6a3 P 7edbfc09954d4b4ab8e4d3df152db05e: Deleting consensus metadata
I20260808 14:32:49.548941 25588 catalog_manager.cc:4841] TS 4f78ca580d054134a91cd7381c1ac124 (127.7.125.66:38209): tablet 98cead73889c4e7a858b1b860862e6a3 (table test-table [id=77abf3bbd7f44ec5b72844d81b76f2bf]) successfully deleted
I20260808 14:32:49.550495 25589 catalog_manager.cc:4841] TS 7edbfc09954d4b4ab8e4d3df152db05e (127.7.125.67:43951): tablet 98cead73889c4e7a858b1b860862e6a3 (table test-table [id=77abf3bbd7f44ec5b72844d81b76f2bf]) successfully deleted
W20260808 14:32:49.551348 25587 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.125.65:40383 (error 108)
I20260808 14:32:49.552161 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 25813
I20260808 14:32:49.552464 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:49.552084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:40383 (local address 127.0.0.1:47118)
0808 14:32:49.552217 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:49.552363 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.65:40383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 14:32:49.554450 25587 catalog_manager.cc:4584] TS 8dc54bdcf41f42ec9618ad14db595f35 (127.7.125.65:40383): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 98cead73889c4e7a858b1b860862e6a3: Network error: Client connection negotiation failed: client connection to 127.7.125.65:40383: connect: Connection refused (error 111)
I20260808 14:32:49.559512 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 25954
I20260808 14:32:49.566763 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 25569
2026-08-08T14:32:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2752 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T14:32:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:32:49Z Disabled control of system clock
I20260808 14:32:49.600948 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:41209
--webserver_interface=127.7.125.126
--webserver_port=0
--builtin_ntp_servers=127.7.125.84:41429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:41209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 14:32:49.711609 26141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:49.711880 26141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:49.711916 26141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:49.716007 26141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:32:49.716070 26141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:49.716086 26141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:49.716104 26141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:32:49.716120 26141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:32:49.720808 26141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:41429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:41209
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:41209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:49.721846 26141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:49.723276 26141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:49.731746 26146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:49.731766 26149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:49.731894 26141 server_base.cc:1034] running on GCE node
W20260808 14:32:49.731760 26147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:49.732683 26141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:49.733786 26141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:49.734999 26141 hybrid_clock.cc:648] HybridClock initialized: now 1786199569734942 us; error 40 us; skew 500 ppm
I20260808 14:32:49.736941 26141 webserver.cc:460] Webserver started at http://127.7.125.126:43181/ using document root <none> and password file <none>
I20260808 14:32:49.737545 26141 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:49.737638 26141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:49.737879 26141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:49.740078 26141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "65f60d219d86477887694e5925b442b6"
format_stamp: "Formatted at 2026-08-08 14:32:49 on dist-test-slave-w206"
server_key: "c934b0038c26aa6e2af0052d83de4722"
server_key_iv: "d58d5127936a46b410b7ebb12a19227e"
server_key_version: "encryptionkey@0"
I20260808 14:32:49.740661 26141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "65f60d219d86477887694e5925b442b6"
format_stamp: "Formatted at 2026-08-08 14:32:49 on dist-test-slave-w206"
server_key: "c934b0038c26aa6e2af0052d83de4722"
server_key_iv: "d58d5127936a46b410b7ebb12a19227e"
server_key_version: "encryptionkey@0"
I20260808 14:32:49.744413 26141 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:32:49.747107 26156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:49.748315 26141 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:32:49.748469 26141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "65f60d219d86477887694e5925b442b6"
format_stamp: "Formatted at 2026-08-08 14:32:49 on dist-test-slave-w206"
server_key: "c934b0038c26aa6e2af0052d83de4722"
server_key_iv: "d58d5127936a46b410b7ebb12a19227e"
server_key_version: "encryptionkey@0"
I20260808 14:32:49.748587 26141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:49.761871 26141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:49.762611 26141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:49.762821 26141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:49.770162 26141 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:41209
I20260808 14:32:49.770166 26210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:41209 every 8 connection(s)
I20260808 14:32:49.771198 26141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:32:49.774477 26211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:49.777063 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 26141
I20260808 14:32:49.777230 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:32:49.777514 7669 external_mini_cluster.cc:1428] Setting key e31e9a29a60c804400da2f07a9f46d08
I20260808 14:32:49.780957 26211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6: Bootstrap starting.
I20260808 14:32:49.783716 26211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:49.784565 26211 log.cc:824] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:49.786940 26211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6: No bootstrap required, opened a new log
I20260808 14:32:49.788240 26214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:49.779097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53186 (local address 127.7.125.126:41209)
0808 14:32:49.779805 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:49.779825 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:49.779873 (+ 48us) server_negotiation.cc:407] Connection header received
0808 14:32:49.780678 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:49.780711 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:49.781210 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:49.781573 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:49.782021 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.783516 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.784461 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:49.785133 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:49.785256 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:49.786195 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:49.786202 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:49.786219 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:32:49.786711 (+ 492us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:49.786782 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:32:49.786808 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:49.786907 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:49.787033 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 14:32:49.787308 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":102,"threads_started":1}
I20260808 14:32:49.791324 26211 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f60d219d86477887694e5925b442b6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 41209 } }
I20260808 14:32:49.791538 26211 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:49.791571 26211 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65f60d219d86477887694e5925b442b6, State: Initialized, Role: FOLLOWER
I20260808 14:32:49.791920 26211 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f60d219d86477887694e5925b442b6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 41209 } }
I20260808 14:32:49.792026 26211 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:32:49.792073 26211 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:32:49.792150 26211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:49.793087 26211 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f60d219d86477887694e5925b442b6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 41209 } }
I20260808 14:32:49.793495 26211 leader_election.cc:302] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65f60d219d86477887694e5925b442b6; no voters:
I20260808 14:32:49.793833 26211 leader_election.cc:288] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:32:49.793959 26216 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:32:49.794219 26216 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6 [term 1 LEADER]: Becoming Leader. State: Replica: 65f60d219d86477887694e5925b442b6, State: Running, Role: LEADER
I20260808 14:32:49.794535 26216 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f60d219d86477887694e5925b442b6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 41209 } }
I20260808 14:32:49.795135 26211 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:32:49.796247 26218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65f60d219d86477887694e5925b442b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f60d219d86477887694e5925b442b6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 41209 } } }
I20260808 14:32:49.796387 26218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6 [sys.catalog]: This master's current role is: LEADER
I20260808 14:32:49.796686 26220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65f60d219d86477887694e5925b442b6. Latest consensus state: current_term: 1 leader_uuid: "65f60d219d86477887694e5925b442b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f60d219d86477887694e5925b442b6" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 41209 } } }
I20260808 14:32:49.796787 26220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6 [sys.catalog]: This master's current role is: LEADER
I20260808 14:32:49.797076 26227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:32:49.799948 26227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:32:49.805327 26227 catalog_manager.cc:1347] Generated new cluster ID: 9f14fcf3873c4b3ca60e3e1df71f49f4
I20260808 14:32:49.805406 26227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:32:49.820760 26227 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:32:49.821841 26227 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:32:49.830626 26227 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6: Generated new TSK 0
I20260808 14:32:49.831389 26227 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:32:49.836392 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:41209
--builtin_ntp_servers=127.7.125.84:41429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 14:32:49.951828 26240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:49.952124 26240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:49.952204 26240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:49.956418 26240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:49.956497 26240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:49.956591 26240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:32:49.961388 26240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:41429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:41209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:49.962488 26240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:49.963975 26240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:49.973695 26248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:49.973701 26246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:49.974100 26240 server_base.cc:1034] running on GCE node
W20260808 14:32:49.973750 26245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:49.974534 26240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:49.975239 26240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:49.976437 26240 hybrid_clock.cc:648] HybridClock initialized: now 1786199569976405 us; error 43 us; skew 500 ppm
I20260808 14:32:49.978263 26240 webserver.cc:460] Webserver started at http://127.7.125.65:44659/ using document root <none> and password file <none>
I20260808 14:32:49.978928 26240 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:49.979055 26240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:49.979293 26240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:49.981406 26240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "3e684af3c6154e49bede585c5983c9a3"
format_stamp: "Formatted at 2026-08-08 14:32:49 on dist-test-slave-w206"
server_key: "27af778d9fba228b31ccf205852ececd"
server_key_iv: "fd3d123d29a83a51215727ec3512930d"
server_key_version: "encryptionkey@0"
I20260808 14:32:49.982008 26240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "3e684af3c6154e49bede585c5983c9a3"
format_stamp: "Formatted at 2026-08-08 14:32:49 on dist-test-slave-w206"
server_key: "27af778d9fba228b31ccf205852ececd"
server_key_iv: "fd3d123d29a83a51215727ec3512930d"
server_key_version: "encryptionkey@0"
I20260808 14:32:49.985924 26240 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260808 14:32:49.988600 26256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:49.989907 26240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 14:32:49.990048 26240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "3e684af3c6154e49bede585c5983c9a3"
format_stamp: "Formatted at 2026-08-08 14:32:49 on dist-test-slave-w206"
server_key: "27af778d9fba228b31ccf205852ececd"
server_key_iv: "fd3d123d29a83a51215727ec3512930d"
server_key_version: "encryptionkey@0"
I20260808 14:32:49.990168 26240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:50.002544 26240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:50.003515 26240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:50.003736 26240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:50.004317 26240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:50.005231 26240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:32:50.005278 26240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:50.005318 26240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:32:50.005333 26240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:50.014564 26240 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:40785
I20260808 14:32:50.014633 26376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:40785 every 8 connection(s)
I20260808 14:32:50.015623 26240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:32:50.024000 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 26240
I20260808 14:32:50.024130 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:32:50.024427 7669 external_mini_cluster.cc:1428] Setting key 0d855da7b59008a11be6d82faf04e4e7
I20260808 14:32:50.025872 26214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:50.017562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:40471 (local address 127.7.125.126:41209)
0808 14:32:50.017723 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:50.017726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:50.018785 (+ 1059us) server_negotiation.cc:407] Connection header received
0808 14:32:50.020247 (+ 1462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:50.020251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:50.020312 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:50.020415 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:50.021576 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.022302 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.023563 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.023911 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.024038 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:50.025355 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:50.025365 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:50.025368 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:32:50.025380 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:50.025399 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:32:50.025407 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:50.025472 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:50.025669 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 14:32:50.025725 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 14:32:50.026753 26379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:50.017941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:41209 (local address 127.7.125.65:40471)
0808 14:32:50.018607 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:50.018647 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:50.020021 (+ 1374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:50.020595 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:50.020602 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:50.020740 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:50.021439 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:50.021457 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.022435 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.022439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:50.023436 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:50.023446 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.024091 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.024098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:50.024308 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:50.025001 (+ 693us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:32:50.025003 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:50.025059 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:32:50.025148 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:50.025501 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:50.025511 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:50.025516 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 14:32:50.025649 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 14:32:50.025958 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":121,"threads_started":1}
I20260808 14:32:50.026757 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:41209
--builtin_ntp_servers=127.7.125.84:41429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 14:32:50.028113 26377 heartbeater.cc:347] Connected to a master server at 127.7.125.126:41209
I20260808 14:32:50.028366 26377 heartbeater.cc:464] Registering TS with master...
I20260808 14:32:50.028885 26377 heartbeater.cc:511] Master 127.7.125.126:41209 requested a full tablet report, sending...
I20260808 14:32:50.030474 26174 ts_manager.cc:194] Registered new tserver with Master: 3e684af3c6154e49bede585c5983c9a3 (127.7.125.65:40785)
I20260808 14:32:50.032390 26174 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.125.65:40471
W20260808 14:32:50.141973 26380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:50.142262 26380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:50.142303 26380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:50.146482 26380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:50.146552 26380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:50.146608 26380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:32:50.151438 26380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:41429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:41209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:50.152632 26380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:50.154120 26380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:50.164474 26386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:50.164505 26389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:50.164686 26380 server_base.cc:1034] running on GCE node
W20260808 14:32:50.164474 26387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:50.165213 26380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:50.165999 26380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:50.167215 26380 hybrid_clock.cc:648] HybridClock initialized: now 1786199570167186 us; error 45 us; skew 500 ppm
I20260808 14:32:50.169322 26380 webserver.cc:460] Webserver started at http://127.7.125.66:38343/ using document root <none> and password file <none>
I20260808 14:32:50.170109 26380 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:50.170222 26380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:50.170462 26380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:50.172715 26380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "ed4758fafc954f03a45546f69f301a28"
format_stamp: "Formatted at 2026-08-08 14:32:50 on dist-test-slave-w206"
server_key: "799a537c03efc30d4653dcd106fa6957"
server_key_iv: "c9195e04033bc7a1db88bfe22507fac2"
server_key_version: "encryptionkey@0"
I20260808 14:32:50.173346 26380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "ed4758fafc954f03a45546f69f301a28"
format_stamp: "Formatted at 2026-08-08 14:32:50 on dist-test-slave-w206"
server_key: "799a537c03efc30d4653dcd106fa6957"
server_key_iv: "c9195e04033bc7a1db88bfe22507fac2"
server_key_version: "encryptionkey@0"
I20260808 14:32:50.177381 26380 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:32:50.180094 26396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:50.181398 26380 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:32:50.181537 26380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "ed4758fafc954f03a45546f69f301a28"
format_stamp: "Formatted at 2026-08-08 14:32:50 on dist-test-slave-w206"
server_key: "799a537c03efc30d4653dcd106fa6957"
server_key_iv: "c9195e04033bc7a1db88bfe22507fac2"
server_key_version: "encryptionkey@0"
I20260808 14:32:50.181656 26380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:50.202742 26380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:50.203572 26380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:50.203807 26380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:50.204474 26380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:50.205716 26380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:32:50.205775 26380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:50.205842 26380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:32:50.205875 26380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:50.215437 26380 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:34925
I20260808 14:32:50.215517 26516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:34925 every 8 connection(s)
I20260808 14:32:50.216457 26380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:32:50.225662 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 26380
I20260808 14:32:50.225826 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:32:50.226123 7669 external_mini_cluster.cc:1428] Setting key 53b0795629c5e9276c79f6fb2cd0437d
I20260808 14:32:50.226279 26214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:50.218392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:35597 (local address 127.7.125.126:41209)
0808 14:32:50.218581 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:50.218586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:50.219766 (+ 1180us) server_negotiation.cc:407] Connection header received
0808 14:32:50.221168 (+ 1402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:50.221171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:50.221221 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:50.221307 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:50.222269 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.222894 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.224097 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.224324 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.224433 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:50.225677 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:50.225681 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:50.225683 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:32:50.225696 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:50.225717 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:32:50.225725 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:50.225831 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:50.226054 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 14:32:50.226127 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 14:32:50.227449 26519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:50.218848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:41209 (local address 127.7.125.66:35597)
0808 14:32:50.219608 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:50.219650 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:50.220960 (+ 1310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:50.221472 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:50.221479 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:50.221564 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:50.222104 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:50.222118 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.223080 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.223083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:50.223975 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:50.223984 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.224505 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.224509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:50.224672 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:50.225314 (+ 642us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:32:50.225349 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:50.225409 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:32:50.225501 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:50.225842 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:50.225856 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:50.225865 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 14:32:50.226040 (+ 175us) client_negotiation.cc:240] Negotiation successful
0808 14:32:50.226348 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":201,"threads_started":1}
I20260808 14:32:50.228657 26517 heartbeater.cc:347] Connected to a master server at 127.7.125.126:41209
I20260808 14:32:50.228960 26517 heartbeater.cc:464] Registering TS with master...
I20260808 14:32:50.229087 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:41209
--builtin_ntp_servers=127.7.125.84:41429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 14:32:50.229689 26517 heartbeater.cc:511] Master 127.7.125.126:41209 requested a full tablet report, sending...
I20260808 14:32:50.230938 26174 ts_manager.cc:194] Registered new tserver with Master: ed4758fafc954f03a45546f69f301a28 (127.7.125.66:34925)
I20260808 14:32:50.231848 26174 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.125.66:35597
W20260808 14:32:50.352583 26520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:50.352957 26520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:50.353051 26520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:50.357079 26520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:50.357167 26520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:50.357273 26520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:32:50.362224 26520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:41429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:41209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:50.363365 26520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:50.364753 26520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:50.374550 26526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:50.374526 26525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:50.374529 26529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:50.374641 26520 server_base.cc:1034] running on GCE node
I20260808 14:32:50.375363 26520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:50.376026 26520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:50.377259 26520 hybrid_clock.cc:648] HybridClock initialized: now 1786199570377246 us; error 31 us; skew 500 ppm
I20260808 14:32:50.379354 26520 webserver.cc:460] Webserver started at http://127.7.125.67:33831/ using document root <none> and password file <none>
I20260808 14:32:50.379979 26520 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:50.380076 26520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:50.380322 26520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:50.382457 26520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "0b98ddc30eb34c62b4281e96baec3032"
format_stamp: "Formatted at 2026-08-08 14:32:50 on dist-test-slave-w206"
server_key: "e7bde8626d9252d572045420314463cb"
server_key_iv: "b0589b462e7b79b60642a45725ff38f4"
server_key_version: "encryptionkey@0"
I20260808 14:32:50.383077 26520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "0b98ddc30eb34c62b4281e96baec3032"
format_stamp: "Formatted at 2026-08-08 14:32:50 on dist-test-slave-w206"
server_key: "e7bde8626d9252d572045420314463cb"
server_key_iv: "b0589b462e7b79b60642a45725ff38f4"
server_key_version: "encryptionkey@0"
I20260808 14:32:50.387166 26520 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 14:32:50.389695 26535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:50.391394 26520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:32:50.391533 26520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "0b98ddc30eb34c62b4281e96baec3032"
format_stamp: "Formatted at 2026-08-08 14:32:50 on dist-test-slave-w206"
server_key: "e7bde8626d9252d572045420314463cb"
server_key_iv: "b0589b462e7b79b60642a45725ff38f4"
server_key_version: "encryptionkey@0"
I20260808 14:32:50.391673 26520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:50.405771 26520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:50.406519 26520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:50.406733 26520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:50.407424 26520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:50.408497 26520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:32:50.408568 26520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:50.408643 26520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:32:50.408695 26520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:50.418531 26520 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:33713
I20260808 14:32:50.418555 26657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:33713 every 8 connection(s)
I20260808 14:32:50.419524 26520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:32:50.428491 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 26520
I20260808 14:32:50.428617 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:32:50.428907 7669 external_mini_cluster.cc:1428] Setting key cd97c24847b878ff582e7e0a1b6e49e1
I20260808 14:32:50.429224 26214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:50.421493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:39335 (local address 127.7.125.126:41209)
0808 14:32:50.421655 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:50.421659 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:50.422815 (+ 1156us) server_negotiation.cc:407] Connection header received
0808 14:32:50.424335 (+ 1520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:50.424338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:50.424399 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:50.424510 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:50.425430 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.425962 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.427173 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.427412 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.427534 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:50.428705 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:50.428709 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:50.428712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:32:50.428734 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:50.428753 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:32:50.428761 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:50.428844 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:50.429003 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 14:32:50.429068 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 14:32:50.430276 26660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:50.421895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:41209 (local address 127.7.125.67:39335)
0808 14:32:50.422555 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:50.422595 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:50.424105 (+ 1510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:50.424692 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:50.424699 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:50.424791 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:50.425273 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:50.425286 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.426109 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.426114 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:50.427031 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:50.427041 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.427581 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.427585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:50.427762 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:50.428433 (+ 671us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:32:50.428435 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:50.428476 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:32:50.428541 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:50.428846 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:50.428858 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:50.428867 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 14:32:50.429003 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 14:32:50.429304 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":101,"threads_started":1}
I20260808 14:32:50.431438 26658 heartbeater.cc:347] Connected to a master server at 127.7.125.126:41209
I20260808 14:32:50.431687 26658 heartbeater.cc:464] Registering TS with master...
I20260808 14:32:50.432237 26658 heartbeater.cc:511] Master 127.7.125.126:41209 requested a full tablet report, sending...
I20260808 14:32:50.433248 26173 ts_manager.cc:194] Registered new tserver with Master: 0b98ddc30eb34c62b4281e96baec3032 (127.7.125.67:33713)
I20260808 14:32:50.433794 26173 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.125.67:39335
I20260808 14:32:50.443382 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:32:50.449182 26214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:50.445568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53192 (local address 127.7.125.126:41209)
0808 14:32:50.445745 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:50.445749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:50.445840 (+ 91us) server_negotiation.cc:407] Connection header received
0808 14:32:50.445943 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:50.445945 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:50.445986 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:50.446055 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:50.446303 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.446808 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.447662 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.447880 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.447957 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:50.448770 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:50.448772 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:50.448775 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:32:50.448784 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:50.448799 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:32:50.448806 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:50.448857 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:50.448954 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 14:32:50.449000 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 14:32:50.452337 26173 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53192:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:32:50.455442 26173 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:32:50.470541 26671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:50.464412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43968 (local address 127.7.125.67:33713)
0808 14:32:50.465297 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:50.465312 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:50.465338 (+ 26us) server_negotiation.cc:407] Connection header received
0808 14:32:50.465423 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:50.465428 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:50.465638 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:50.466189 (+ 551us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:50.466331 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.467498 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.468913 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.469761 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.469880 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:50.470109 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:50.470227 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 14:32:50.470340 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":761,"thread_start_us":168,"threads_started":1}
I20260808 14:32:50.470578 26672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:50.464975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:40785 (local address 127.0.0.1:48934)
0808 14:32:50.465964 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:50.465976 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:50.466066 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:50.466566 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:50.466569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:50.466588 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:50.466666 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:50.466672 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.468007 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.468010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:50.469176 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:50.469183 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.470179 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.470185 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:50.470304 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:50.470323 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 14:32:50.470388 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 14:32:50.470458 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":934,"mutex_wait_us":44,"thread_start_us":100,"threads_started":1}
I20260808 14:32:50.470579 26670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:50.464206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:33713 (local address 127.0.0.1:43968)
0808 14:32:50.464814 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:50.464865 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:50.464985 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:50.465832 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:50.465838 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:50.465867 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:50.466040 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:50.466049 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.467622 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.467626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:50.468782 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:50.468792 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.469899 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.469903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:50.470008 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:50.470087 (+ 79us) client_negotiation.cc:769] Sending connection context
0808 14:32:50.470200 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 14:32:50.470327 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"mutex_wait_us":9,"thread_start_us":92,"threads_started":1}
I20260808 14:32:50.471053 26673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:50.464962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39826 (local address 127.7.125.66:34925)
0808 14:32:50.465549 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:50.465562 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:50.465585 (+ 23us) server_negotiation.cc:407] Connection header received
0808 14:32:50.465659 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:50.465666 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:50.465853 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:50.466009 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:50.466270 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.467265 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.469087 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.469880 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.470497 (+ 617us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:50.470698 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:50.470755 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 14:32:50.470874 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":122,"threads_started":1}
I20260808 14:32:50.471278 26669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:50.464416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:34925 (local address 127.0.0.1:39826)
0808 14:32:50.465149 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:50.465164 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:50.465256 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:50.465982 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:50.465985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:50.466004 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:50.466133 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:50.466143 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.467428 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.467431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:50.468961 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:50.468971 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.470016 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.470019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:50.470166 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:50.470245 (+ 79us) client_negotiation.cc:769] Sending connection context
0808 14:32:50.470307 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 14:32:50.470377 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":186,"thread_start_us":125,"threads_started":1}
I20260808 14:32:50.471450 26674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:50.465396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48934 (local address 127.7.125.65:40785)
0808 14:32:50.465821 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:50.465834 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:50.466164 (+ 330us) server_negotiation.cc:407] Connection header received
0808 14:32:50.466242 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:50.466247 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:50.466411 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:50.466559 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:50.466788 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.467883 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.469285 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.469797 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.469922 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:50.470117 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:50.470876 (+ 759us) server_negotiation.cc:299] Negotiation successful
0808 14:32:50.471001 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":109,"threads_started":1}
I20260808 14:32:50.473913 26587 tablet_service.cc:1467] Processing CreateTablet for tablet 2206643e799d47219b86f64d7c3a3b3c (DEFAULT_TABLE table=test-table [id=15fbf302a8a34be4b54ebf395c0eee44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:32:50.474134 26448 tablet_service.cc:1467] Processing CreateTablet for tablet 2206643e799d47219b86f64d7c3a3b3c (DEFAULT_TABLE table=test-table [id=15fbf302a8a34be4b54ebf395c0eee44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:32:50.474135 26307 tablet_service.cc:1467] Processing CreateTablet for tablet 2206643e799d47219b86f64d7c3a3b3c (DEFAULT_TABLE table=test-table [id=15fbf302a8a34be4b54ebf395c0eee44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:32:50.475047 26587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2206643e799d47219b86f64d7c3a3b3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:50.475186 26307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2206643e799d47219b86f64d7c3a3b3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:50.475186 26448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2206643e799d47219b86f64d7c3a3b3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:50.481261 26675 tablet_bootstrap.cc:492] T 2206643e799d47219b86f64d7c3a3b3c P 0b98ddc30eb34c62b4281e96baec3032: Bootstrap starting.
I20260808 14:32:50.482900 26679 tablet_bootstrap.cc:492] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28: Bootstrap starting.
I20260808 14:32:50.484011 26675 tablet_bootstrap.cc:654] T 2206643e799d47219b86f64d7c3a3b3c P 0b98ddc30eb34c62b4281e96baec3032: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:50.484740 26675 log.cc:824] T 2206643e799d47219b86f64d7c3a3b3c P 0b98ddc30eb34c62b4281e96baec3032: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:50.484890 26679 tablet_bootstrap.cc:654] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:50.485493 26678 tablet_bootstrap.cc:492] T 2206643e799d47219b86f64d7c3a3b3c P 3e684af3c6154e49bede585c5983c9a3: Bootstrap starting.
I20260808 14:32:50.485805 26679 log.cc:824] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:50.487130 26675 tablet_bootstrap.cc:492] T 2206643e799d47219b86f64d7c3a3b3c P 0b98ddc30eb34c62b4281e96baec3032: No bootstrap required, opened a new log
I20260808 14:32:50.487339 26675 ts_tablet_manager.cc:1397] T 2206643e799d47219b86f64d7c3a3b3c P 0b98ddc30eb34c62b4281e96baec3032: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 14:32:50.487594 26678 tablet_bootstrap.cc:654] T 2206643e799d47219b86f64d7c3a3b3c P 3e684af3c6154e49bede585c5983c9a3: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:50.488324 26678 log.cc:824] T 2206643e799d47219b86f64d7c3a3b3c P 3e684af3c6154e49bede585c5983c9a3: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:50.488574 26679 tablet_bootstrap.cc:492] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28: No bootstrap required, opened a new log
I20260808 14:32:50.488760 26679 ts_tablet_manager.cc:1397] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260808 14:32:50.490013 26678 tablet_bootstrap.cc:492] T 2206643e799d47219b86f64d7c3a3b3c P 3e684af3c6154e49bede585c5983c9a3: No bootstrap required, opened a new log
I20260808 14:32:50.490201 26678 ts_tablet_manager.cc:1397] T 2206643e799d47219b86f64d7c3a3b3c P 3e684af3c6154e49bede585c5983c9a3: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 14:32:50.492332 26675 raft_consensus.cc:348] T 2206643e799d47219b86f64d7c3a3b3c P 0b98ddc30eb34c62b4281e96baec3032 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b98ddc30eb34c62b4281e96baec3032" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 33713 } } peers { permanent_uuid: "ed4758fafc954f03a45546f69f301a28" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34925 } } peers { permanent_uuid: "3e684af3c6154e49bede585c5983c9a3" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 40785 } }
I20260808 14:32:50.492624 26675 raft_consensus.cc:374] T 2206643e799d47219b86f64d7c3a3b3c P 0b98ddc30eb34c62b4281e96baec3032 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:50.492681 26675 raft_consensus.cc:729] T 2206643e799d47219b86f64d7c3a3b3c P 0b98ddc30eb34c62b4281e96baec3032 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b98ddc30eb34c62b4281e96baec3032, State: Initialized, Role: FOLLOWER
I20260808 14:32:50.492594 26679 raft_consensus.cc:348] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b98ddc30eb34c62b4281e96baec3032" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 33713 } } peers { permanent_uuid: "ed4758fafc954f03a45546f69f301a28" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34925 } } peers { permanent_uuid: "3e684af3c6154e49bede585c5983c9a3" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 40785 } }
I20260808 14:32:50.492831 26679 raft_consensus.cc:374] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:50.492901 26679 raft_consensus.cc:729] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed4758fafc954f03a45546f69f301a28, State: Initialized, Role: FOLLOWER
I20260808 14:32:50.493165 26675 consensus_queue.cc:260] T 2206643e799d47219b86f64d7c3a3b3c P 0b98ddc30eb34c62b4281e96baec3032 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b98ddc30eb34c62b4281e96baec3032" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 33713 } } peers { permanent_uuid: "ed4758fafc954f03a45546f69f301a28" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34925 } } peers { permanent_uuid: "3e684af3c6154e49bede585c5983c9a3" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 40785 } }
I20260808 14:32:50.493263 26679 consensus_queue.cc:260] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b98ddc30eb34c62b4281e96baec3032" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 33713 } } peers { permanent_uuid: "ed4758fafc954f03a45546f69f301a28" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34925 } } peers { permanent_uuid: "3e684af3c6154e49bede585c5983c9a3" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 40785 } }
I20260808 14:32:50.493882 26658 heartbeater.cc:503] Master 127.7.125.126:41209 was elected leader, sending a full tablet report...
I20260808 14:32:50.494277 26675 ts_tablet_manager.cc:1428] T 2206643e799d47219b86f64d7c3a3b3c P 0b98ddc30eb34c62b4281e96baec3032: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 14:32:50.494277 26679 ts_tablet_manager.cc:1428] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260808 14:32:50.494655 26517 heartbeater.cc:503] Master 127.7.125.126:41209 was elected leader, sending a full tablet report...
I20260808 14:32:50.495110 26678 raft_consensus.cc:348] T 2206643e799d47219b86f64d7c3a3b3c P 3e684af3c6154e49bede585c5983c9a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b98ddc30eb34c62b4281e96baec3032" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 33713 } } peers { permanent_uuid: "ed4758fafc954f03a45546f69f301a28" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34925 } } peers { permanent_uuid: "3e684af3c6154e49bede585c5983c9a3" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 40785 } }
I20260808 14:32:50.495365 26678 raft_consensus.cc:374] T 2206643e799d47219b86f64d7c3a3b3c P 3e684af3c6154e49bede585c5983c9a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:50.495415 26678 raft_consensus.cc:729] T 2206643e799d47219b86f64d7c3a3b3c P 3e684af3c6154e49bede585c5983c9a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e684af3c6154e49bede585c5983c9a3, State: Initialized, Role: FOLLOWER
I20260808 14:32:50.495929 26678 consensus_queue.cc:260] T 2206643e799d47219b86f64d7c3a3b3c P 3e684af3c6154e49bede585c5983c9a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b98ddc30eb34c62b4281e96baec3032" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 33713 } } peers { permanent_uuid: "ed4758fafc954f03a45546f69f301a28" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34925 } } peers { permanent_uuid: "3e684af3c6154e49bede585c5983c9a3" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 40785 } }
I20260808 14:32:50.496714 26377 heartbeater.cc:503] Master 127.7.125.126:41209 was elected leader, sending a full tablet report...
I20260808 14:32:50.497110 26678 ts_tablet_manager.cc:1428] T 2206643e799d47219b86f64d7c3a3b3c P 3e684af3c6154e49bede585c5983c9a3: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260808 14:32:50.516829 26378 tablet.cc:2369] T 2206643e799d47219b86f64d7c3a3b3c P 3e684af3c6154e49bede585c5983c9a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:32:50.652019 26684 raft_consensus.cc:482] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:32:50.652268 26684 raft_consensus.cc:504] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b98ddc30eb34c62b4281e96baec3032" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 33713 } } peers { permanent_uuid: "ed4758fafc954f03a45546f69f301a28" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34925 } } peers { permanent_uuid: "3e684af3c6154e49bede585c5983c9a3" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 40785 } }
I20260808 14:32:50.653553 26684 leader_election.cc:288] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b98ddc30eb34c62b4281e96baec3032 (127.7.125.67:33713), 3e684af3c6154e49bede585c5983c9a3 (127.7.125.65:40785)
I20260808 14:32:50.657755 26519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:50.653872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:33713 (local address 127.7.125.66:58859)
0808 14:32:50.654037 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:50.654072 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:50.654186 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:50.654604 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:50.654607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:50.654623 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:50.654721 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:50.654726 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.655564 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.655568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:50.656613 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:50.656622 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.657286 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.657290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:50.657405 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:50.657436 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 14:32:50.657508 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 14:32:50.657602 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":27}
I20260808 14:32:50.658143 26671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:50.654009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:58859 (local address 127.7.125.67:33713)
0808 14:32:50.654159 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:50.654181 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:50.654199 (+ 18us) server_negotiation.cc:407] Connection header received
0808 14:32:50.654312 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:50.654315 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:50.654368 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:50.654465 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:50.654843 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.655442 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.656739 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.657155 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.657291 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:50.657343 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:50.657947 (+ 604us) server_negotiation.cc:299] Negotiation successful
0808 14:32:50.658024 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 14:32:50.658833 26686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:50.654311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:40785 (local address 127.7.125.66:59551)
0808 14:32:50.654710 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:50.654724 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:50.654865 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:50.655187 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:50.655190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:50.655209 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:50.655345 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:50.655351 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.656206 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.656211 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:50.657241 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:50.657252 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.658200 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.658203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:50.658481 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:50.658503 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 14:32:50.658608 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 14:32:50.658685 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":4,"thread_start_us":140,"threads_started":1}
I20260808 14:32:50.658845 26674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:50.654375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:59551 (local address 127.7.125.65:40785)
0808 14:32:50.654535 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:50.654542 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:50.654823 (+ 281us) server_negotiation.cc:407] Connection header received
0808 14:32:50.655019 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:50.655023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:50.655080 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:50.655172 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:50.655478 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.656088 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.657399 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.657967 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.658090 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:50.658127 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:50.658630 (+ 503us) server_negotiation.cc:299] Negotiation successful
0808 14:32:50.658713 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 14:32:50.658783 26609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2206643e799d47219b86f64d7c3a3b3c" candidate_uuid: "ed4758fafc954f03a45546f69f301a28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b98ddc30eb34c62b4281e96baec3032" is_pre_election: true
I20260808 14:32:50.659194 26609 raft_consensus.cc:2454] T 2206643e799d47219b86f64d7c3a3b3c P 0b98ddc30eb34c62b4281e96baec3032 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed4758fafc954f03a45546f69f301a28 in term 0.
I20260808 14:32:50.659574 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2206643e799d47219b86f64d7c3a3b3c" candidate_uuid: "ed4758fafc954f03a45546f69f301a28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e684af3c6154e49bede585c5983c9a3" is_pre_election: true
I20260808 14:32:50.659796 26400 leader_election.cc:302] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b98ddc30eb34c62b4281e96baec3032, ed4758fafc954f03a45546f69f301a28; no voters:
I20260808 14:32:50.659900 26329 raft_consensus.cc:2454] T 2206643e799d47219b86f64d7c3a3b3c P 3e684af3c6154e49bede585c5983c9a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed4758fafc954f03a45546f69f301a28 in term 0.
I20260808 14:32:50.660120 26684 raft_consensus.cc:2781] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:32:50.660223 26684 raft_consensus.cc:482] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:32:50.660280 26684 raft_consensus.cc:3037] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:50.661392 26684 raft_consensus.cc:504] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b98ddc30eb34c62b4281e96baec3032" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 33713 } } peers { permanent_uuid: "ed4758fafc954f03a45546f69f301a28" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34925 } } peers { permanent_uuid: "3e684af3c6154e49bede585c5983c9a3" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 40785 } }
I20260808 14:32:50.661782 26684 leader_election.cc:288] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28 [CANDIDATE]: Term 1 election: Requested vote from peers 0b98ddc30eb34c62b4281e96baec3032 (127.7.125.67:33713), 3e684af3c6154e49bede585c5983c9a3 (127.7.125.65:40785)
I20260808 14:32:50.662190 26609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2206643e799d47219b86f64d7c3a3b3c" candidate_uuid: "ed4758fafc954f03a45546f69f301a28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b98ddc30eb34c62b4281e96baec3032"
I20260808 14:32:50.662207 26329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2206643e799d47219b86f64d7c3a3b3c" candidate_uuid: "ed4758fafc954f03a45546f69f301a28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e684af3c6154e49bede585c5983c9a3"
I20260808 14:32:50.662326 26609 raft_consensus.cc:3037] T 2206643e799d47219b86f64d7c3a3b3c P 0b98ddc30eb34c62b4281e96baec3032 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:50.662326 26329 raft_consensus.cc:3037] T 2206643e799d47219b86f64d7c3a3b3c P 3e684af3c6154e49bede585c5983c9a3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:50.663668 26329 raft_consensus.cc:2454] T 2206643e799d47219b86f64d7c3a3b3c P 3e684af3c6154e49bede585c5983c9a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed4758fafc954f03a45546f69f301a28 in term 1.
I20260808 14:32:50.663668 26609 raft_consensus.cc:2454] T 2206643e799d47219b86f64d7c3a3b3c P 0b98ddc30eb34c62b4281e96baec3032 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed4758fafc954f03a45546f69f301a28 in term 1.
I20260808 14:32:50.663977 26400 leader_election.cc:302] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e684af3c6154e49bede585c5983c9a3, ed4758fafc954f03a45546f69f301a28; no voters:
I20260808 14:32:50.664145 26684 raft_consensus.cc:2781] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:32:50.664402 26684 raft_consensus.cc:686] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28 [term 1 LEADER]: Becoming Leader. State: Replica: ed4758fafc954f03a45546f69f301a28, State: Running, Role: LEADER
I20260808 14:32:50.664752 26684 consensus_queue.cc:237] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b98ddc30eb34c62b4281e96baec3032" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 33713 } } peers { permanent_uuid: "ed4758fafc954f03a45546f69f301a28" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34925 } } peers { permanent_uuid: "3e684af3c6154e49bede585c5983c9a3" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 40785 } }
I20260808 14:32:50.668013 26174 catalog_manager.cc:5495] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28 reported cstate change: term changed from 0 to 1, leader changed from <none> to ed4758fafc954f03a45546f69f301a28 (127.7.125.66). New cstate: current_term: 1 leader_uuid: "ed4758fafc954f03a45546f69f301a28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b98ddc30eb34c62b4281e96baec3032" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 33713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed4758fafc954f03a45546f69f301a28" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 34925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e684af3c6154e49bede585c5983c9a3" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 40785 } health_report { overall_health: UNKNOWN } } }
W20260808 14:32:50.670485 26659 tablet.cc:2369] T 2206643e799d47219b86f64d7c3a3b3c P 0b98ddc30eb34c62b4281e96baec3032: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:32:50.711514 26673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:50.708170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39828 (local address 127.7.125.66:34925)
0808 14:32:50.708304 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:50.708308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:50.708320 (+ 12us) server_negotiation.cc:407] Connection header received
0808 14:32:50.708395 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:50.708397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:50.708456 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:50.708547 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:32:50.708797 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.709297 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.710137 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:50.710339 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:50.710412 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:50.710676 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:32:50.711167 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:32:50.711260 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:50.711357 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 14:32:50.711404 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 14:32:50.717419 26329 raft_consensus.cc:1261] T 2206643e799d47219b86f64d7c3a3b3c P 3e684af3c6154e49bede585c5983c9a3 [term 1 FOLLOWER]: Refusing update from remote peer ed4758fafc954f03a45546f69f301a28: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:32:50.717419 26609 raft_consensus.cc:1261] T 2206643e799d47219b86f64d7c3a3b3c P 0b98ddc30eb34c62b4281e96baec3032 [term 1 FOLLOWER]: Refusing update from remote peer ed4758fafc954f03a45546f69f301a28: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260808 14:32:50.717648 26518 tablet.cc:2369] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:32:50.718163 26684 consensus_queue.cc:1035] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b98ddc30eb34c62b4281e96baec3032" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 33713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:32:50.718389 26687 consensus_queue.cc:1035] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e684af3c6154e49bede585c5983c9a3" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:32:50.734895 26174 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:32:50.735100 26174 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:32:50.737628 26174 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 65f60d219d86477887694e5925b442b6: Sending DeleteTablet for 3 replicas of tablet 2206643e799d47219b86f64d7c3a3b3c
I20260808 14:32:50.738643 26307 tablet_service.cc:1514] Processing DeleteTablet for tablet 2206643e799d47219b86f64d7c3a3b3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:32:50 UTC) from {username='slave'} at 127.0.0.1:48934
I20260808 14:32:50.738633 26448 tablet_service.cc:1514] Processing DeleteTablet for tablet 2206643e799d47219b86f64d7c3a3b3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:32:50 UTC) from {username='slave'} at 127.0.0.1:39826
I20260808 14:32:50.739238 26587 tablet_service.cc:1514] Processing DeleteTablet for tablet 2206643e799d47219b86f64d7c3a3b3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:32:50 UTC) from {username='slave'} at 127.0.0.1:43968
I20260808 14:32:50.739771 26703 tablet_replica.cc:330] T 2206643e799d47219b86f64d7c3a3b3c P 0b98ddc30eb34c62b4281e96baec3032: stopping tablet replica
I20260808 14:32:50.740060 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 26240
I20260808 14:32:50.740125 26703 raft_consensus.cc:2229] T 2206643e799d47219b86f64d7c3a3b3c P 0b98ddc30eb34c62b4281e96baec3032 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:32:50.740447 26703 raft_consensus.cc:2258] T 2206643e799d47219b86f64d7c3a3b3c P 0b98ddc30eb34c62b4281e96baec3032 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:32:50.740819 26702 tablet_replica.cc:330] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28: stopping tablet replica
I20260808 14:32:50.741091 26702 raft_consensus.cc:2229] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:32:50.741472 26702 raft_consensus.cc:2258] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:32:50.743100 26702 ts_tablet_manager.cc:1905] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:32:50.743106 26703 ts_tablet_manager.cc:1905] T 2206643e799d47219b86f64d7c3a3b3c P 0b98ddc30eb34c62b4281e96baec3032: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:32:50.745793 26703 ts_tablet_manager.cc:1918] T 2206643e799d47219b86f64d7c3a3b3c P 0b98ddc30eb34c62b4281e96baec3032: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:32:50.745864 26702 ts_tablet_manager.cc:1918] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:32:50.745914 26703 log.cc:1196] T 2206643e799d47219b86f64d7c3a3b3c P 0b98ddc30eb34c62b4281e96baec3032: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/2206643e799d47219b86f64d7c3a3b3c
I20260808 14:32:50.745955 26702 log.cc:1196] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/2206643e799d47219b86f64d7c3a3b3c
I20260808 14:32:50.746208 26703 ts_tablet_manager.cc:1939] T 2206643e799d47219b86f64d7c3a3b3c P 0b98ddc30eb34c62b4281e96baec3032: Deleting consensus metadata
I20260808 14:32:50.746269 26702 ts_tablet_manager.cc:1939] T 2206643e799d47219b86f64d7c3a3b3c P ed4758fafc954f03a45546f69f301a28: Deleting consensus metadata
I20260808 14:32:50.747381 26160 catalog_manager.cc:4841] TS 0b98ddc30eb34c62b4281e96baec3032 (127.7.125.67:33713): tablet 2206643e799d47219b86f64d7c3a3b3c (table test-table [id=15fbf302a8a34be4b54ebf395c0eee44]) successfully deleted
I20260808 14:32:50.747385 26158 catalog_manager.cc:4841] TS ed4758fafc954f03a45546f69f301a28 (127.7.125.66:34925): tablet 2206643e799d47219b86f64d7c3a3b3c (table test-table [id=15fbf302a8a34be4b54ebf395c0eee44]) successfully deleted
W20260808 14:32:50.754153 26160 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.125.65:40785 (error 108)
I20260808 14:32:50.754251 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 26380
I20260808 14:32:50.757930 26669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:50.757618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:40785 (local address 127.0.0.1:48942)
0808 14:32:50.757758 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:50.757844 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.65:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 14:32:50.758121 26160 catalog_manager.cc:4584] TS 3e684af3c6154e49bede585c5983c9a3 (127.7.125.65:40785): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2206643e799d47219b86f64d7c3a3b3c: Network error: Client connection negotiation failed: client connection to 127.7.125.65:40785: connect: Connection refused (error 111)
I20260808 14:32:50.761562 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 26520
I20260808 14:32:50.768563 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 26141
2026-08-08T14:32:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1200 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:32:50 dist-test-slave-w206 krb5kdc[26707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:32:50 dist-test-slave-w206 krb5kdc[26707](info): set up 2 sockets
Aug 08 14:32:50 dist-test-slave-w206 krb5kdc[26707](info): commencing operation
krb5kdc: starting...
W20260808 14:32:51.851960 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:32:51 dist-test-slave-w206 krb5kdc[26707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:32:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:32:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:32:51.986069 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:45735
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:45735
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:32:52.099599 26722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:52.099962 26722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:52.100054 26722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:52.104146 26722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:32:52.104277 26722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:52.104333 26722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:52.104369 26722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:32:52.104403 26722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:32:52.109800 26722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:45735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:45735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:52.110991 26722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:52.112076 26722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:52.120846 26727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:52.120996 26728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:52.120846 26730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:52.121852 26722 server_base.cc:1034] running on GCE node
I20260808 14:32:52.122623 26722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:52.124285 26722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:52.125516 26722 hybrid_clock.cc:648] HybridClock initialized: now 1786199572125485 us; error 49 us; skew 500 ppm
Aug 08 14:32:52 dist-test-slave-w206 krb5kdc[26707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199572, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:32:52.129331 26722 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:32:52.130519 26722 webserver.cc:460] Webserver started at http://127.7.125.126:43451/ using document root <none> and password file <none>
I20260808 14:32:52.131210 26722 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:52.131281 26722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:52.131523 26722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:52.133689 26722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "7f13919644694f329f3e3de256742a9b"
format_stamp: "Formatted at 2026-08-08 14:32:52 on dist-test-slave-w206"
server_key: "d50a94ea4fb63e815dfbd118012fc720"
server_key_iv: "05160fd25c627704a33584abddba2d80"
server_key_version: "encryptionkey@0"
I20260808 14:32:52.134318 26722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "7f13919644694f329f3e3de256742a9b"
format_stamp: "Formatted at 2026-08-08 14:32:52 on dist-test-slave-w206"
server_key: "d50a94ea4fb63e815dfbd118012fc720"
server_key_iv: "05160fd25c627704a33584abddba2d80"
server_key_version: "encryptionkey@0"
I20260808 14:32:52.138305 26722 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260808 14:32:52.140997 26737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:52.142299 26722 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 14:32:52.142462 26722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "7f13919644694f329f3e3de256742a9b"
format_stamp: "Formatted at 2026-08-08 14:32:52 on dist-test-slave-w206"
server_key: "d50a94ea4fb63e815dfbd118012fc720"
server_key_iv: "05160fd25c627704a33584abddba2d80"
server_key_version: "encryptionkey@0"
I20260808 14:32:52.142589 26722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:52.197548 26722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:52.201293 26722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:52.201515 26722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:52.209277 26722 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:45735
I20260808 14:32:52.209304 26791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:45735 every 8 connection(s)
I20260808 14:32:52.210335 26722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:32:52.213243 26793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:52.213996 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 26722
I20260808 14:32:52.214161 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:32:52.214457 7669 external_mini_cluster.cc:1428] Setting key ff20bec0659c14ab77d1fb322b05ed0a
I20260808 14:32:52.219798 26793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b: Bootstrap starting.
I20260808 14:32:52.222206 26793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:52.223093 26793 log.cc:824] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:32:52 dist-test-slave-w206 krb5kdc[26707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:32:52.225337 26793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b: No bootstrap required, opened a new log
I20260808 14:32:52.229156 26793 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f13919644694f329f3e3de256742a9b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 45735 } }
I20260808 14:32:52.229444 26793 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:52.229535 26793 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f13919644694f329f3e3de256742a9b, State: Initialized, Role: FOLLOWER
I20260808 14:32:52.229836 26796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:52.216058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36654 (local address 127.7.125.126:45735)
0808 14:32:52.216533 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:52.216548 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:52.216584 (+ 36us) server_negotiation.cc:407] Connection header received
0808 14:32:52.217248 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:52.217265 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:52.217599 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:52.217940 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:52.218847 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:52.220076 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:52.221000 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:52.221500 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:52.221697 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:52.224647 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:52.224653 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:52.224669 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:52.224698 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:52.226649 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:52.227208 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:52.227215 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:52.227221 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:52.227346 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:52.227641 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:52.227645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:52.227647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:52.228067 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:52.228302 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:52.228558 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 14:32:52.228876 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":144,"threads_started":1}
I20260808 14:32:52.230037 26793 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f13919644694f329f3e3de256742a9b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 45735 } }
I20260808 14:32:52.230203 26793 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:32:52.230271 26793 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:32:52.230368 26793 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:52.231674 26793 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f13919644694f329f3e3de256742a9b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 45735 } }
I20260808 14:32:52.232040 26793 leader_election.cc:302] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f13919644694f329f3e3de256742a9b; no voters:
I20260808 14:32:52.232406 26793 leader_election.cc:288] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:32:52.232698 26799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:32:52.232906 26799 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b [term 1 LEADER]: Becoming Leader. State: Replica: 7f13919644694f329f3e3de256742a9b, State: Running, Role: LEADER
I20260808 14:32:52.233202 26799 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f13919644694f329f3e3de256742a9b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 45735 } }
I20260808 14:32:52.233757 26793 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:32:52.234462 26799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f13919644694f329f3e3de256742a9b. Latest consensus state: current_term: 1 leader_uuid: "7f13919644694f329f3e3de256742a9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f13919644694f329f3e3de256742a9b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 45735 } } }
I20260808 14:32:52.234594 26799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b [sys.catalog]: This master's current role is: LEADER
I20260808 14:32:52.235309 26802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f13919644694f329f3e3de256742a9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f13919644694f329f3e3de256742a9b" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 45735 } } }
I20260808 14:32:52.235581 26802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b [sys.catalog]: This master's current role is: LEADER
W20260808 14:32:52.239022 26816 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:32:52.239104 26816 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:32:52.239184 26808 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:32:52.240147 26808 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:32:52.245532 26808 catalog_manager.cc:1347] Generated new cluster ID: 9dc9950617234017a11519e1cacb032d
I20260808 14:32:52.245609 26808 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:32:52.278153 26808 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:32:52.279287 26808 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:32:52.287168 26808 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b: Generated new TSK 0
I20260808 14:32:52.287923 26808 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.125.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
I20260808 14:32:52.331250 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:45735
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:32:52.445542 26823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:52.445906 26823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:52.445982 26823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:52.450241 26823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:52.450300 26823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:52.450415 26823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:32:52.455574 26823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:45735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:52.456563 26823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:52.457549 26823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:52.467525 26828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:52.467484 26829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:52.467531 26831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:52.467715 26823 server_base.cc:1034] running on GCE node
I20260808 14:32:52.468214 26823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:52.468935 26823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:52.470131 26823 hybrid_clock.cc:648] HybridClock initialized: now 1786199572470108 us; error 38 us; skew 500 ppm
Aug 08 14:32:52 dist-test-slave-w206 krb5kdc[26707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199572, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:32:52.472889 26823 init.cc:377] Logged in from keytab as kudu/127.7.125.65@KRBTEST.COM (short username kudu)
I20260808 14:32:52.473731 26823 webserver.cc:460] Webserver started at http://127.7.125.65:34999/ using document root <none> and password file <none>
I20260808 14:32:52.474287 26823 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:52.474401 26823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:52.474632 26823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:52.476703 26823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "1c700a93c3574538899fdfdfa374885d"
format_stamp: "Formatted at 2026-08-08 14:32:52 on dist-test-slave-w206"
server_key: "15bfa219745a764ec03f0d79410852f9"
server_key_iv: "48dda1cbd503a1032ff88962b0ec6d58"
server_key_version: "encryptionkey@0"
I20260808 14:32:52.477276 26823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "1c700a93c3574538899fdfdfa374885d"
format_stamp: "Formatted at 2026-08-08 14:32:52 on dist-test-slave-w206"
server_key: "15bfa219745a764ec03f0d79410852f9"
server_key_iv: "48dda1cbd503a1032ff88962b0ec6d58"
server_key_version: "encryptionkey@0"
I20260808 14:32:52.481194 26823 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260808 14:32:52.483675 26839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:52.484953 26823 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 14:32:52.485103 26823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "1c700a93c3574538899fdfdfa374885d"
format_stamp: "Formatted at 2026-08-08 14:32:52 on dist-test-slave-w206"
server_key: "15bfa219745a764ec03f0d79410852f9"
server_key_iv: "48dda1cbd503a1032ff88962b0ec6d58"
server_key_version: "encryptionkey@0"
I20260808 14:32:52.485216 26823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:52.496323 26823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:52.499346 26823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:52.499542 26823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:52.500130 26823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:52.500963 26823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:32:52.501034 26823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:52.501106 26823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:32:52.501159 26823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:52.510233 26823 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:43395
I20260808 14:32:52.510262 26959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:43395 every 8 connection(s)
I20260808 14:32:52.511202 26823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:32:52.519168 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 26823
I20260808 14:32:52.519346 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:32:52.520066 7669 external_mini_cluster.cc:1428] Setting key 3f9588335e705c64ea1527536b2278d3
Aug 08 14:32:52 dist-test-slave-w206 krb5kdc[26707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199572, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:32:52.525903 26796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:52.513003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:40259 (local address 127.7.125.126:45735)
0808 14:32:52.513144 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:52.513148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:52.514191 (+ 1043us) server_negotiation.cc:407] Connection header received
0808 14:32:52.515139 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:52.515142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:52.515193 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:52.515270 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:52.516586 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:52.517122 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:52.518463 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:52.518677 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:52.518754 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:52.521896 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:52.521899 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:52.521901 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:52.521924 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:52.523390 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:52.524049 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:52.524056 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:52.524058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:52.524159 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:52.524543 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:52.524548 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:52.524551 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:52.524813 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:52.524934 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:52.525496 (+ 562us) server_negotiation.cc:299] Negotiation successful
0808 14:32:52.525648 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 14:32:52.526672 26962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:52.513308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:45735 (local address 127.7.125.65:40259)
0808 14:32:52.514019 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:52.514061 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:52.514848 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:52.515482 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:52.515489 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:52.515889 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:52.516441 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:52.516455 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:52.517279 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:52.517282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:52.518352 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:52.518362 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:52.518849 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:52.518856 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:52.519081 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:52.519873 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:32:52.519896 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:52.521726 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:52.523507 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:52.523517 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:52.523535 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:52.523927 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:52.524298 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:52.524301 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:52.524302 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:52.524444 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:52.524966 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:52.524972 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:52.525233 (+ 261us) client_negotiation.cc:769] Sending connection context
0808 14:32:52.525410 (+ 177us) client_negotiation.cc:240] Negotiation successful
0808 14:32:52.525694 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":111,"threads_started":1}
I20260808 14:32:52.528080 26960 heartbeater.cc:347] Connected to a master server at 127.7.125.126:45735
I20260808 14:32:52.528383 26960 heartbeater.cc:464] Registering TS with master...
I20260808 14:32:52.528893 26960 heartbeater.cc:511] Master 127.7.125.126:45735 requested a full tablet report, sending...
I20260808 14:32:52.530457 26754 ts_manager.cc:194] Registered new tserver with Master: 1c700a93c3574538899fdfdfa374885d (127.7.125.65:43395)
WARNING: no policy specified for kudu/127.7.125.66@KRBTEST.COM; defaulting to no policy
I20260808 14:32:52.531699 26754 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:40259
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
I20260808 14:32:52.550735 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:45735
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:32:52.666069 26966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:52.666378 26966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:52.666453 26966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:52.670683 26966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:52.670780 26966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:52.670872 26966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:32:52.676296 26966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:45735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:52.677469 26966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:52.678503 26966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:52.688570 26972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:52.688556 26974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:52.688560 26971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:52.688602 26966 server_base.cc:1034] running on GCE node
I20260808 14:32:52.689298 26966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:52.689944 26966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:52.691082 26966 hybrid_clock.cc:648] HybridClock initialized: now 1786199572691049 us; error 44 us; skew 500 ppm
Aug 08 14:32:52 dist-test-slave-w206 krb5kdc[26707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199572, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:32:52.693990 26966 init.cc:377] Logged in from keytab as kudu/127.7.125.66@KRBTEST.COM (short username kudu)
I20260808 14:32:52.694898 26966 webserver.cc:460] Webserver started at http://127.7.125.66:37057/ using document root <none> and password file <none>
I20260808 14:32:52.695493 26966 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:52.695556 26966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:52.695804 26966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:52.697860 26966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "c0a0e82fa88243a3b050d47009f298f6"
format_stamp: "Formatted at 2026-08-08 14:32:52 on dist-test-slave-w206"
server_key: "a0a7b5245066effaf3fa8aae90916fb2"
server_key_iv: "0167841598f68716c8617406133fdff5"
server_key_version: "encryptionkey@0"
I20260808 14:32:52.698426 26966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "c0a0e82fa88243a3b050d47009f298f6"
format_stamp: "Formatted at 2026-08-08 14:32:52 on dist-test-slave-w206"
server_key: "a0a7b5245066effaf3fa8aae90916fb2"
server_key_iv: "0167841598f68716c8617406133fdff5"
server_key_version: "encryptionkey@0"
I20260808 14:32:52.702267 26966 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:32:52.704723 26981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:52.705986 26966 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 14:32:52.706140 26966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "c0a0e82fa88243a3b050d47009f298f6"
format_stamp: "Formatted at 2026-08-08 14:32:52 on dist-test-slave-w206"
server_key: "a0a7b5245066effaf3fa8aae90916fb2"
server_key_iv: "0167841598f68716c8617406133fdff5"
server_key_version: "encryptionkey@0"
I20260808 14:32:52.706259 26966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:52.730836 26966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:52.734323 26966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:52.734534 26966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:52.735282 26966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:52.736182 26966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:32:52.736243 26966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:52.736295 26966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:32:52.736334 26966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:52.745020 26966 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:38601
I20260808 14:32:52.745044 27105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:38601 every 8 connection(s)
I20260808 14:32:52.745961 26966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:32:52.748103 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 26966
I20260808 14:32:52.748236 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:32:52.748513 7669 external_mini_cluster.cc:1428] Setting key 8a8d9f0e7a4cc5d0d9d0a084babb4598
Aug 08 14:32:52 dist-test-slave-w206 krb5kdc[26707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199572, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.125.67@KRBTEST.COM; defaulting to no policy
I20260808 14:32:52.760640 26796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:52.747750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:40781 (local address 127.7.125.126:45735)
0808 14:32:52.747889 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:52.747894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:52.748813 (+ 919us) server_negotiation.cc:407] Connection header received
0808 14:32:52.749873 (+ 1060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:52.749876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:52.749927 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:52.750011 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:52.751204 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:52.752043 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:52.753879 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:52.754096 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:52.754174 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:52.757008 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:52.757012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:52.757014 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:52.757037 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:52.758427 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:52.758958 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:52.758964 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:52.758966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:52.759114 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:52.759478 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:52.759482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:52.759484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:52.759678 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:52.759789 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:52.760315 (+ 526us) server_negotiation.cc:299] Negotiation successful
0808 14:32:52.760459 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 14:32:52.761464 27108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:52.748082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:45735 (local address 127.7.125.66:40781)
0808 14:32:52.748596 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:52.748672 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:52.749680 (+ 1008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:52.750162 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:52.750168 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:52.750516 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:52.751059 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:52.751075 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:52.752222 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:52.752226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:52.753739 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:52.753754 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:52.754259 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:52.754266 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:52.754506 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:52.755259 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:32:52.755275 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:52.756821 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:52.758558 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:52.758564 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:52.758578 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:52.758840 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:52.759211 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:52.759215 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:52.759217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:52.759389 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:52.759788 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:52.759795 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:52.760073 (+ 278us) client_negotiation.cc:769] Sending connection context
0808 14:32:52.760271 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 14:32:52.760609 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":154,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.67@KRBTEST.COM" created.
I20260808 14:32:52.762825 27106 heartbeater.cc:347] Connected to a master server at 127.7.125.126:45735
I20260808 14:32:52.763140 27106 heartbeater.cc:464] Registering TS with master...
I20260808 14:32:52.763850 27106 heartbeater.cc:511] Master 127.7.125.126:45735 requested a full tablet report, sending...
I20260808 14:32:52.765158 26754 ts_manager.cc:194] Registered new tserver with Master: c0a0e82fa88243a3b050d47009f298f6 (127.7.125.66:38601)
I20260808 14:32:52.765829 26754 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:40781
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
I20260808 14:32:52.781985 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:45735
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:32:52.900161 27111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:52.900450 27111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:52.900498 27111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:52.904546 27111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:52.904633 27111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:52.904704 27111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:32:52.910175 27111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:45735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:52.911345 27111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:52.912364 27111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:32:52.922401 27111 server_base.cc:1034] running on GCE node
W20260808 14:32:52.922353 27117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:52.922353 27120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:52.922370 27118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:52.923054 27111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:52.923755 27111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:52.924930 27111 hybrid_clock.cc:648] HybridClock initialized: now 1786199572924898 us; error 45 us; skew 500 ppm
Aug 08 14:32:52 dist-test-slave-w206 krb5kdc[26707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199572, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:32:52.927835 27111 init.cc:377] Logged in from keytab as kudu/127.7.125.67@KRBTEST.COM (short username kudu)
I20260808 14:32:52.928756 27111 webserver.cc:460] Webserver started at http://127.7.125.67:37741/ using document root <none> and password file <none>
I20260808 14:32:52.929335 27111 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:52.929401 27111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:52.929653 27111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:52.931824 27111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "bb298ae43592429bb9de924935555ffb"
format_stamp: "Formatted at 2026-08-08 14:32:52 on dist-test-slave-w206"
server_key: "6cbeff3107e5525c57a2f524a56169ba"
server_key_iv: "d224ae3a57048e715a3c337257f12549"
server_key_version: "encryptionkey@0"
I20260808 14:32:52.932417 27111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "bb298ae43592429bb9de924935555ffb"
format_stamp: "Formatted at 2026-08-08 14:32:52 on dist-test-slave-w206"
server_key: "6cbeff3107e5525c57a2f524a56169ba"
server_key_iv: "d224ae3a57048e715a3c337257f12549"
server_key_version: "encryptionkey@0"
I20260808 14:32:52.936553 27111 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:32:52.938954 27128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:52.940326 27111 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:32:52.940419 27111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "bb298ae43592429bb9de924935555ffb"
format_stamp: "Formatted at 2026-08-08 14:32:52 on dist-test-slave-w206"
server_key: "6cbeff3107e5525c57a2f524a56169ba"
server_key_iv: "d224ae3a57048e715a3c337257f12549"
server_key_version: "encryptionkey@0"
I20260808 14:32:52.940500 27111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:52.969549 27111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:52.972628 27111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:52.972798 27111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:52.973347 27111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:52.974231 27111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:32:52.974306 27111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:52.974383 27111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:32:52.974431 27111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:52.983803 27111 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:33945
I20260808 14:32:52.983816 27249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:33945 every 8 connection(s)
I20260808 14:32:52.984716 27111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:32:52.989699 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 27111
I20260808 14:32:52.989868 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:32:52.990170 7669 external_mini_cluster.cc:1428] Setting key 4694d51b2dcf78767d88df0e8f4b4390
Aug 08 14:32:52 dist-test-slave-w206 krb5kdc[26707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199572, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:32:52.999022 26796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:52.986612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:41547 (local address 127.7.125.126:45735)
0808 14:32:52.986752 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:52.986756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:52.987547 (+ 791us) server_negotiation.cc:407] Connection header received
0808 14:32:52.988899 (+ 1352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:52.988903 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:52.988990 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:52.989085 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:52.990438 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:52.991215 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:52.992805 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:52.993029 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:52.993109 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:52.995609 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:52.995613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:52.995616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:52.995659 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:52.996930 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:52.997451 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:52.997455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:52.997457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:52.997543 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:52.997894 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:52.997897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:52.997899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:52.998090 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:52.998198 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:52.998688 (+ 490us) server_negotiation.cc:299] Negotiation successful
0808 14:32:52.998811 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 14:32:52.999801 27252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:52.986929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:45735 (local address 127.7.125.67:41547)
0808 14:32:52.987394 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:52.987438 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:52.988682 (+ 1244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:52.989255 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:52.989269 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:52.989631 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:52.990203 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:52.990218 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:52.991362 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:52.991365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:52.992697 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:52.992707 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:52.993157 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:52.993160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:52.993332 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:52.993886 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:32:52.993902 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:52.995403 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:52.997051 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:52.997057 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:52.997072 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:52.997334 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:52.997643 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:52.997646 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:52.997648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:52.997789 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:52.998197 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:52.998204 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:52.998453 (+ 249us) client_negotiation.cc:769] Sending connection context
0808 14:32:52.998645 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 14:32:52.998968 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":118,"threads_started":1}
I20260808 14:32:53.000959 27250 heartbeater.cc:347] Connected to a master server at 127.7.125.126:45735
I20260808 14:32:53.001217 27250 heartbeater.cc:464] Registering TS with master...
I20260808 14:32:53.001735 27250 heartbeater.cc:511] Master 127.7.125.126:45735 requested a full tablet report, sending...
I20260808 14:32:53.002776 26754 ts_manager.cc:194] Registered new tserver with Master: bb298ae43592429bb9de924935555ffb (127.7.125.67:33945)
I20260808 14:32:53.003582 26754 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:41547
I20260808 14:32:53.004375 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:32:53.014726 26796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:53.006390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36664 (local address 127.7.125.126:45735)
0808 14:32:53.006505 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:53.006509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:53.006673 (+ 164us) server_negotiation.cc:407] Connection header received
0808 14:32:53.006769 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:53.006771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:53.006807 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:53.006872 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:53.007663 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.008245 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.009106 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.009370 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.009447 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:53.011286 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:53.011289 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:53.011291 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:53.011312 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:53.012902 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:53.013646 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:53.013649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:53.013650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:53.013723 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:53.013973 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:53.013976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:53.013977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:53.014169 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:53.014283 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:53.014534 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 14:32:53.014613 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 14:32:53.017964 26754 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:32:53.021270 26754 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:32:53.037284 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:53.030158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:33945 (local address 127.0.0.1:43606)
0808 14:32:53.031129 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:53.031186 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:53.031324 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:53.032030 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:53.032034 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:53.032053 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:53.032231 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:53.032239 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.034332 (+ 2093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.034336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:53.035510 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:53.035517 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.036611 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.036615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:53.036717 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:53.036809 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 14:32:53.036907 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 14:32:53.037078 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":882,"mutex_wait_us":69,"spinlock_wait_cycles":6656,"thread_start_us":120,"threads_started":1}
I20260808 14:32:53.037493 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:53.031265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43606 (local address 127.7.125.67:33945)
0808 14:32:53.031636 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:53.031647 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:53.031666 (+ 19us) server_negotiation.cc:407] Connection header received
0808 14:32:53.031730 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:53.031734 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:53.031905 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:53.032041 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:53.033020 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.034185 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.035638 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.036461 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.037043 (+ 582us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:53.037187 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:53.037238 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 14:32:53.037336 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":168,"threads_started":1}
I20260808 14:32:53.037787 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:53.030774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39292 (local address 127.7.125.66:38601)
0808 14:32:53.032269 (+ 1495us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:53.032278 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:53.032323 (+ 45us) server_negotiation.cc:407] Connection header received
0808 14:32:53.032406 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:53.032412 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:53.032611 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:53.032786 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:53.033092 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.034282 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.035890 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.036713 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.036813 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:53.036952 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:53.037498 (+ 546us) server_negotiation.cc:299] Negotiation successful
0808 14:32:53.037669 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1393,"thread_start_us":145,"threads_started":1}
I20260808 14:32:53.037885 27263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:53.030020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:43395 (local address 127.0.0.1:37602)
0808 14:32:53.031130 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:53.031186 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:53.031324 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:53.031963 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:53.031971 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:53.031999 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:53.032231 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:53.032239 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.033751 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.033754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:53.035320 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:53.035330 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.036576 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.036580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:53.036730 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:53.036811 (+ 81us) client_negotiation.cc:769] Sending connection context
0808 14:32:53.036946 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 14:32:53.037737 (+ 791us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":908,"mutex_wait_us":78,"spinlock_wait_cycles":8064,"thread_start_us":75,"threads_started":1}
I20260808 14:32:53.038195 27264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:53.030555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:38601 (local address 127.0.0.1:39292)
0808 14:32:53.031720 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:53.031734 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:53.031817 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:53.032795 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:53.032813 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:53.032831 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:53.032953 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:53.032958 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.034455 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.034479 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:53.035753 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:53.035766 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.037014 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.037017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:53.037107 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:53.037128 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 14:32:53.037196 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 14:32:53.037267 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1103,"thread_start_us":102,"threads_started":1}
I20260808 14:32:53.038551 27262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:53.030140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37602 (local address 127.7.125.65:43395)
0808 14:32:53.030949 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:53.030957 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:53.031485 (+ 528us) server_negotiation.cc:407] Connection header received
0808 14:32:53.031574 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:53.031580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:53.031794 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:53.031949 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:53.032373 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.033609 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.035451 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.036425 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.036583 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:53.036752 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:53.036986 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 14:32:53.038378 (+ 1392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":92,"threads_started":1}
I20260808 14:32:53.041003 27032 tablet_service.cc:1467] Processing CreateTablet for tablet f21ce359d2ed47e99ef4dcef92c06331 (DEFAULT_TABLE table=test-table [id=7345bc9ee02c49948c51d726073c5150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:32:53.041031 26891 tablet_service.cc:1467] Processing CreateTablet for tablet f21ce359d2ed47e99ef4dcef92c06331 (DEFAULT_TABLE table=test-table [id=7345bc9ee02c49948c51d726073c5150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:32:53.041007 27179 tablet_service.cc:1467] Processing CreateTablet for tablet f21ce359d2ed47e99ef4dcef92c06331 (DEFAULT_TABLE table=test-table [id=7345bc9ee02c49948c51d726073c5150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:32:53.042098 27179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f21ce359d2ed47e99ef4dcef92c06331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:53.042096 26891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f21ce359d2ed47e99ef4dcef92c06331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:53.042096 27032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f21ce359d2ed47e99ef4dcef92c06331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:53.049618 27268 tablet_bootstrap.cc:492] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d: Bootstrap starting.
I20260808 14:32:53.050848 27269 tablet_bootstrap.cc:492] T f21ce359d2ed47e99ef4dcef92c06331 P bb298ae43592429bb9de924935555ffb: Bootstrap starting.
I20260808 14:32:53.051292 27271 tablet_bootstrap.cc:492] T f21ce359d2ed47e99ef4dcef92c06331 P c0a0e82fa88243a3b050d47009f298f6: Bootstrap starting.
I20260808 14:32:53.052878 27268 tablet_bootstrap.cc:654] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:53.053081 27271 tablet_bootstrap.cc:654] T f21ce359d2ed47e99ef4dcef92c06331 P c0a0e82fa88243a3b050d47009f298f6: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:53.053190 27269 tablet_bootstrap.cc:654] T f21ce359d2ed47e99ef4dcef92c06331 P bb298ae43592429bb9de924935555ffb: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:53.054216 27268 log.cc:824] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:53.054369 27271 log.cc:824] T f21ce359d2ed47e99ef4dcef92c06331 P c0a0e82fa88243a3b050d47009f298f6: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:53.054437 27269 log.cc:824] T f21ce359d2ed47e99ef4dcef92c06331 P bb298ae43592429bb9de924935555ffb: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:53.056421 27269 tablet_bootstrap.cc:492] T f21ce359d2ed47e99ef4dcef92c06331 P bb298ae43592429bb9de924935555ffb: No bootstrap required, opened a new log
I20260808 14:32:53.056469 27268 tablet_bootstrap.cc:492] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d: No bootstrap required, opened a new log
I20260808 14:32:53.056512 27271 tablet_bootstrap.cc:492] T f21ce359d2ed47e99ef4dcef92c06331 P c0a0e82fa88243a3b050d47009f298f6: No bootstrap required, opened a new log
I20260808 14:32:53.056675 27268 ts_tablet_manager.cc:1397] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 14:32:53.056659 27269 ts_tablet_manager.cc:1397] T f21ce359d2ed47e99ef4dcef92c06331 P bb298ae43592429bb9de924935555ffb: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260808 14:32:53.056720 27271 ts_tablet_manager.cc:1397] T f21ce359d2ed47e99ef4dcef92c06331 P c0a0e82fa88243a3b050d47009f298f6: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 14:32:53.059940 27269 raft_consensus.cc:348] T f21ce359d2ed47e99ef4dcef92c06331 P bb298ae43592429bb9de924935555ffb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c700a93c3574538899fdfdfa374885d" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43395 } } peers { permanent_uuid: "bb298ae43592429bb9de924935555ffb" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 33945 } } peers { permanent_uuid: "c0a0e82fa88243a3b050d47009f298f6" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38601 } }
I20260808 14:32:53.060237 27269 raft_consensus.cc:374] T f21ce359d2ed47e99ef4dcef92c06331 P bb298ae43592429bb9de924935555ffb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:53.060294 27269 raft_consensus.cc:729] T f21ce359d2ed47e99ef4dcef92c06331 P bb298ae43592429bb9de924935555ffb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb298ae43592429bb9de924935555ffb, State: Initialized, Role: FOLLOWER
I20260808 14:32:53.060844 27269 consensus_queue.cc:260] T f21ce359d2ed47e99ef4dcef92c06331 P bb298ae43592429bb9de924935555ffb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c700a93c3574538899fdfdfa374885d" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43395 } } peers { permanent_uuid: "bb298ae43592429bb9de924935555ffb" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 33945 } } peers { permanent_uuid: "c0a0e82fa88243a3b050d47009f298f6" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38601 } }
I20260808 14:32:53.060776 27271 raft_consensus.cc:348] T f21ce359d2ed47e99ef4dcef92c06331 P c0a0e82fa88243a3b050d47009f298f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c700a93c3574538899fdfdfa374885d" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43395 } } peers { permanent_uuid: "bb298ae43592429bb9de924935555ffb" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 33945 } } peers { permanent_uuid: "c0a0e82fa88243a3b050d47009f298f6" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38601 } }
I20260808 14:32:53.061056 27271 raft_consensus.cc:374] T f21ce359d2ed47e99ef4dcef92c06331 P c0a0e82fa88243a3b050d47009f298f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:53.061127 27271 raft_consensus.cc:729] T f21ce359d2ed47e99ef4dcef92c06331 P c0a0e82fa88243a3b050d47009f298f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0a0e82fa88243a3b050d47009f298f6, State: Initialized, Role: FOLLOWER
I20260808 14:32:53.061272 27268 raft_consensus.cc:348] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c700a93c3574538899fdfdfa374885d" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43395 } } peers { permanent_uuid: "bb298ae43592429bb9de924935555ffb" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 33945 } } peers { permanent_uuid: "c0a0e82fa88243a3b050d47009f298f6" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38601 } }
I20260808 14:32:53.061503 27268 raft_consensus.cc:374] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:53.061558 27268 raft_consensus.cc:729] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c700a93c3574538899fdfdfa374885d, State: Initialized, Role: FOLLOWER
I20260808 14:32:53.061520 27271 consensus_queue.cc:260] T f21ce359d2ed47e99ef4dcef92c06331 P c0a0e82fa88243a3b050d47009f298f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c700a93c3574538899fdfdfa374885d" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43395 } } peers { permanent_uuid: "bb298ae43592429bb9de924935555ffb" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 33945 } } peers { permanent_uuid: "c0a0e82fa88243a3b050d47009f298f6" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38601 } }
I20260808 14:32:53.061940 27268 consensus_queue.cc:260] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c700a93c3574538899fdfdfa374885d" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43395 } } peers { permanent_uuid: "bb298ae43592429bb9de924935555ffb" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 33945 } } peers { permanent_uuid: "c0a0e82fa88243a3b050d47009f298f6" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38601 } }
I20260808 14:32:53.062048 27269 ts_tablet_manager.cc:1428] T f21ce359d2ed47e99ef4dcef92c06331 P bb298ae43592429bb9de924935555ffb: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 14:32:53.062103 27250 heartbeater.cc:503] Master 127.7.125.126:45735 was elected leader, sending a full tablet report...
I20260808 14:32:53.062408 27271 ts_tablet_manager.cc:1428] T f21ce359d2ed47e99ef4dcef92c06331 P c0a0e82fa88243a3b050d47009f298f6: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 14:32:53.062595 26960 heartbeater.cc:503] Master 127.7.125.126:45735 was elected leader, sending a full tablet report...
I20260808 14:32:53.062716 27106 heartbeater.cc:503] Master 127.7.125.126:45735 was elected leader, sending a full tablet report...
I20260808 14:32:53.063815 27268 ts_tablet_manager.cc:1428] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 14:32:53.123517 27278 raft_consensus.cc:482] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:32:53.123786 27278 raft_consensus.cc:504] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c700a93c3574538899fdfdfa374885d" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43395 } } peers { permanent_uuid: "bb298ae43592429bb9de924935555ffb" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 33945 } } peers { permanent_uuid: "c0a0e82fa88243a3b050d47009f298f6" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38601 } }
I20260808 14:32:53.124967 27278 leader_election.cc:288] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb298ae43592429bb9de924935555ffb (127.7.125.67:33945), c0a0e82fa88243a3b050d47009f298f6 (127.7.125.66:38601)
I20260808 14:32:53.129765 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:53.125419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:38939 (local address 127.7.125.66:38601)
0808 14:32:53.125555 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:53.125560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:53.125818 (+ 258us) server_negotiation.cc:407] Connection header received
0808 14:32:53.126007 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:53.126010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:53.126056 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:53.126139 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:53.126415 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.126942 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.128412 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.129100 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.129264 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:53.129336 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:53.129544 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 14:32:53.129634 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 14:32:53.129874 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:53.125405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:39007 (local address 127.7.125.67:33945)
0808 14:32:53.125548 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:53.125552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:53.126002 (+ 450us) server_negotiation.cc:407] Connection header received
0808 14:32:53.126229 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:53.126232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:53.126281 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:53.126383 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:53.126739 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.127558 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.128656 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.129223 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.129346 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:53.129394 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:53.129682 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 14:32:53.129752 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 14:32:53.129806 27279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:53.125277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:38601 (local address 127.7.125.65:38939)
0808 14:32:53.125711 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:53.125723 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:53.125865 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:53.126160 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:53.126162 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:53.126177 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:53.126302 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:53.126307 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.127088 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.127092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:53.128248 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:53.128259 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.129252 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.129257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:53.129382 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:53.129414 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 14:32:53.129512 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 14:32:53.129591 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":140,"threads_started":1}
I20260808 14:32:53.130149 27280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:53.125277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:33945 (local address 127.7.125.65:39007)
0808 14:32:53.125841 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:53.125857 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:53.126045 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:53.126415 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:53.126419 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:53.126437 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:53.126577 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:53.126584 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.127684 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.127687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:53.128541 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:53.128550 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.129419 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.129424 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:53.129558 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:53.129582 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 14:32:53.129667 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 14:32:53.129733 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":43,"thread_start_us":131,"threads_started":1}
I20260808 14:32:53.130708 27053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f21ce359d2ed47e99ef4dcef92c06331" candidate_uuid: "1c700a93c3574538899fdfdfa374885d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0a0e82fa88243a3b050d47009f298f6" is_pre_election: true
I20260808 14:32:53.130934 27201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f21ce359d2ed47e99ef4dcef92c06331" candidate_uuid: "1c700a93c3574538899fdfdfa374885d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb298ae43592429bb9de924935555ffb" is_pre_election: true
I20260808 14:32:53.131112 27053 raft_consensus.cc:2454] T f21ce359d2ed47e99ef4dcef92c06331 P c0a0e82fa88243a3b050d47009f298f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c700a93c3574538899fdfdfa374885d in term 0.
I20260808 14:32:53.131318 27201 raft_consensus.cc:2454] T f21ce359d2ed47e99ef4dcef92c06331 P bb298ae43592429bb9de924935555ffb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c700a93c3574538899fdfdfa374885d in term 0.
I20260808 14:32:53.131671 26842 leader_election.cc:302] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c700a93c3574538899fdfdfa374885d, c0a0e82fa88243a3b050d47009f298f6; no voters:
I20260808 14:32:53.131990 27278 raft_consensus.cc:2781] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:32:53.132092 27278 raft_consensus.cc:482] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:32:53.132149 27278 raft_consensus.cc:3037] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:53.133255 27278 raft_consensus.cc:504] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c700a93c3574538899fdfdfa374885d" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43395 } } peers { permanent_uuid: "bb298ae43592429bb9de924935555ffb" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 33945 } } peers { permanent_uuid: "c0a0e82fa88243a3b050d47009f298f6" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38601 } }
I20260808 14:32:53.133677 27278 leader_election.cc:288] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d [CANDIDATE]: Term 1 election: Requested vote from peers bb298ae43592429bb9de924935555ffb (127.7.125.67:33945), c0a0e82fa88243a3b050d47009f298f6 (127.7.125.66:38601)
I20260808 14:32:53.134018 27053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f21ce359d2ed47e99ef4dcef92c06331" candidate_uuid: "1c700a93c3574538899fdfdfa374885d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0a0e82fa88243a3b050d47009f298f6"
I20260808 14:32:53.134035 27201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f21ce359d2ed47e99ef4dcef92c06331" candidate_uuid: "1c700a93c3574538899fdfdfa374885d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb298ae43592429bb9de924935555ffb"
I20260808 14:32:53.134155 27201 raft_consensus.cc:3037] T f21ce359d2ed47e99ef4dcef92c06331 P bb298ae43592429bb9de924935555ffb [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:53.134164 27053 raft_consensus.cc:3037] T f21ce359d2ed47e99ef4dcef92c06331 P c0a0e82fa88243a3b050d47009f298f6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:53.135396 27053 raft_consensus.cc:2454] T f21ce359d2ed47e99ef4dcef92c06331 P c0a0e82fa88243a3b050d47009f298f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c700a93c3574538899fdfdfa374885d in term 1.
I20260808 14:32:53.135404 27201 raft_consensus.cc:2454] T f21ce359d2ed47e99ef4dcef92c06331 P bb298ae43592429bb9de924935555ffb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c700a93c3574538899fdfdfa374885d in term 1.
I20260808 14:32:53.135692 26842 leader_election.cc:302] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c700a93c3574538899fdfdfa374885d, c0a0e82fa88243a3b050d47009f298f6; no voters:
I20260808 14:32:53.135885 27278 raft_consensus.cc:2781] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:32:53.136158 27278 raft_consensus.cc:686] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d [term 1 LEADER]: Becoming Leader. State: Replica: 1c700a93c3574538899fdfdfa374885d, State: Running, Role: LEADER
I20260808 14:32:53.136535 27278 consensus_queue.cc:237] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c700a93c3574538899fdfdfa374885d" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43395 } } peers { permanent_uuid: "bb298ae43592429bb9de924935555ffb" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 33945 } } peers { permanent_uuid: "c0a0e82fa88243a3b050d47009f298f6" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38601 } }
I20260808 14:32:53.139714 26753 catalog_manager.cc:5495] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c700a93c3574538899fdfdfa374885d (127.7.125.65). New cstate: current_term: 1 leader_uuid: "1c700a93c3574538899fdfdfa374885d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c700a93c3574538899fdfdfa374885d" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb298ae43592429bb9de924935555ffb" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 33945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0a0e82fa88243a3b050d47009f298f6" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38601 } health_report { overall_health: UNKNOWN } } }
I20260808 14:32:53.162206 27262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:53.158546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37610 (local address 127.7.125.65:43395)
0808 14:32:53.158696 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:53.158703 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:53.158721 (+ 18us) server_negotiation.cc:407] Connection header received
0808 14:32:53.158810 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:53.158812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:53.158879 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:53.159016 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:32:53.159284 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.159798 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.160680 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.160909 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.161016 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:53.161349 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:32:53.161770 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:32:53.161942 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:53.162013 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 14:32:53.162053 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 14:32:53.168280 27053 raft_consensus.cc:1261] T f21ce359d2ed47e99ef4dcef92c06331 P c0a0e82fa88243a3b050d47009f298f6 [term 1 FOLLOWER]: Refusing update from remote peer 1c700a93c3574538899fdfdfa374885d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:32:53.168280 27201 raft_consensus.cc:1261] T f21ce359d2ed47e99ef4dcef92c06331 P bb298ae43592429bb9de924935555ffb [term 1 FOLLOWER]: Refusing update from remote peer 1c700a93c3574538899fdfdfa374885d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:32:53.169072 27281 consensus_queue.cc:1035] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0a0e82fa88243a3b050d47009f298f6" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:32:53.169278 27278 consensus_queue.cc:1035] T f21ce359d2ed47e99ef4dcef92c06331 P 1c700a93c3574538899fdfdfa374885d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb298ae43592429bb9de924935555ffb" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 33945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:32:53.180318 27286 mvcc.cc:205] Tried to move back new op lower bound from 7316273451689562112 to 7316273451570155520. Current Snapshot: MvccSnapshot[applied={T|T < 7316273451689562112}]
I20260808 14:32:53.186650 26753 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:32:53.186861 26753 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:32:53.189751 26753 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7f13919644694f329f3e3de256742a9b: Sending DeleteTablet for 3 replicas of tablet f21ce359d2ed47e99ef4dcef92c06331
I20260808 14:32:53.190604 26891 tablet_service.cc:1514] Processing DeleteTablet for tablet f21ce359d2ed47e99ef4dcef92c06331 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:32:53 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:37602
I20260808 14:32:53.190637 27179 tablet_service.cc:1514] Processing DeleteTablet for tablet f21ce359d2ed47e99ef4dcef92c06331 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:32:53 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:43606
I20260808 14:32:53.191244 27032 tablet_service.cc:1514] Processing DeleteTablet for tablet f21ce359d2ed47e99ef4dcef92c06331 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:32:53 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:39292
I20260808 14:32:53.191787 27296 tablet_replica.cc:330] T f21ce359d2ed47e99ef4dcef92c06331 P c0a0e82fa88243a3b050d47009f298f6: stopping tablet replica
I20260808 14:32:53.191960 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 26823
I20260808 14:32:53.194705 27295 tablet_replica.cc:330] T f21ce359d2ed47e99ef4dcef92c06331 P bb298ae43592429bb9de924935555ffb: stopping tablet replica
I20260808 14:32:53.194797 27296 raft_consensus.cc:2229] T f21ce359d2ed47e99ef4dcef92c06331 P c0a0e82fa88243a3b050d47009f298f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:32:53.195042 27295 raft_consensus.cc:2229] T f21ce359d2ed47e99ef4dcef92c06331 P bb298ae43592429bb9de924935555ffb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:32:53.195168 27296 raft_consensus.cc:2258] T f21ce359d2ed47e99ef4dcef92c06331 P c0a0e82fa88243a3b050d47009f298f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:32:53.195328 27295 raft_consensus.cc:2258] T f21ce359d2ed47e99ef4dcef92c06331 P bb298ae43592429bb9de924935555ffb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:32:53.196919 27296 ts_tablet_manager.cc:1905] T f21ce359d2ed47e99ef4dcef92c06331 P c0a0e82fa88243a3b050d47009f298f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:32:53.196919 27295 ts_tablet_manager.cc:1905] T f21ce359d2ed47e99ef4dcef92c06331 P bb298ae43592429bb9de924935555ffb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:32:53.199828 27296 ts_tablet_manager.cc:1918] T f21ce359d2ed47e99ef4dcef92c06331 P c0a0e82fa88243a3b050d47009f298f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:32:53.199828 27295 ts_tablet_manager.cc:1918] T f21ce359d2ed47e99ef4dcef92c06331 P bb298ae43592429bb9de924935555ffb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:32:53.199959 27296 log.cc:1196] T f21ce359d2ed47e99ef4dcef92c06331 P c0a0e82fa88243a3b050d47009f298f6: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/f21ce359d2ed47e99ef4dcef92c06331
I20260808 14:32:53.199959 27295 log.cc:1196] T f21ce359d2ed47e99ef4dcef92c06331 P bb298ae43592429bb9de924935555ffb: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/f21ce359d2ed47e99ef4dcef92c06331
I20260808 14:32:53.200332 27295 ts_tablet_manager.cc:1939] T f21ce359d2ed47e99ef4dcef92c06331 P bb298ae43592429bb9de924935555ffb: Deleting consensus metadata
I20260808 14:32:53.200332 27296 ts_tablet_manager.cc:1939] T f21ce359d2ed47e99ef4dcef92c06331 P c0a0e82fa88243a3b050d47009f298f6: Deleting consensus metadata
W20260808 14:32:53.201239 26741 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.125.65:43395 (error 108)
I20260808 14:32:53.201740 26740 catalog_manager.cc:4841] TS c0a0e82fa88243a3b050d47009f298f6 (127.7.125.66:38601): tablet f21ce359d2ed47e99ef4dcef92c06331 (table test-table [id=7345bc9ee02c49948c51d726073c5150]) successfully deleted
I20260808 14:32:53.201776 26738 catalog_manager.cc:4841] TS bb298ae43592429bb9de924935555ffb (127.7.125.67:33945): tablet f21ce359d2ed47e99ef4dcef92c06331 (table test-table [id=7345bc9ee02c49948c51d726073c5150]) successfully deleted
I20260808 14:32:53.202390 27263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:53.202083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:43395 (local address 127.0.0.1:37614)
0808 14:32:53.202206 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:53.202273 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.65:43395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 14:32:53.202486 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 26966
W20260808 14:32:53.202572 26741 catalog_manager.cc:4584] TS 1c700a93c3574538899fdfdfa374885d (127.7.125.65:43395): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f21ce359d2ed47e99ef4dcef92c06331: Network error: Client connection negotiation failed: client connection to 127.7.125.65:43395: connect: Connection refused (error 111)
I20260808 14:32:53.210824 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 27111
I20260808 14:32:53.217694 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 26722
2026-08-08T14:32:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2451 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T14:32:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:32:53Z Disabled control of system clock
I20260808 14:32:53.252552 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:36381
--webserver_interface=127.7.125.126
--webserver_port=0
--builtin_ntp_servers=127.7.125.84:41517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:36381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 14:32:53.369036 27301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:53.369315 27301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:53.369351 27301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:53.373338 27301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:32:53.373404 27301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:53.373420 27301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:53.373436 27301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:32:53.373451 27301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:32:53.378396 27301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:41517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:36381
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:36381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:53.379547 27301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:53.381002 27301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:53.389585 27307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:53.389693 27309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:53.389585 27306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:53.389874 27301 server_base.cc:1034] running on GCE node
I20260808 14:32:53.390487 27301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:53.391677 27301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:53.392876 27301 hybrid_clock.cc:648] HybridClock initialized: now 1786199573392855 us; error 31 us; skew 500 ppm
I20260808 14:32:53.394757 27301 webserver.cc:460] Webserver started at http://127.7.125.126:35525/ using document root <none> and password file <none>
I20260808 14:32:53.395418 27301 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:53.395517 27301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:53.395761 27301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:53.397881 27301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "4bb3d656378c47bd83852cc9759f30e5"
format_stamp: "Formatted at 2026-08-08 14:32:53 on dist-test-slave-w206"
server_key: "00937d4271599ea57ef5ae92a9def574"
server_key_iv: "7e7321f2c10d18236eb2955292ae7513"
server_key_version: "encryptionkey@0"
I20260808 14:32:53.398492 27301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "4bb3d656378c47bd83852cc9759f30e5"
format_stamp: "Formatted at 2026-08-08 14:32:53 on dist-test-slave-w206"
server_key: "00937d4271599ea57ef5ae92a9def574"
server_key_iv: "7e7321f2c10d18236eb2955292ae7513"
server_key_version: "encryptionkey@0"
I20260808 14:32:53.402944 27301 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:32:53.405766 27315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:53.407251 27301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:32:53.407411 27301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "4bb3d656378c47bd83852cc9759f30e5"
format_stamp: "Formatted at 2026-08-08 14:32:53 on dist-test-slave-w206"
server_key: "00937d4271599ea57ef5ae92a9def574"
server_key_iv: "7e7321f2c10d18236eb2955292ae7513"
server_key_version: "encryptionkey@0"
I20260808 14:32:53.407537 27301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:53.418141 27301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:53.418874 27301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:53.419106 27301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:53.426409 27301 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:36381
I20260808 14:32:53.426481 27369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:36381 every 8 connection(s)
I20260808 14:32:53.427469 27301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:32:53.429019 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 27301
I20260808 14:32:53.429183 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:32:53.429533 7669 external_mini_cluster.cc:1428] Setting key 2ab957685b73b48f54df84b883f4df5e
I20260808 14:32:53.431526 27370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:53.439225 27370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5: Bootstrap starting.
I20260808 14:32:53.441243 27373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:53.431027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34950 (local address 127.7.125.126:36381)
0808 14:32:53.431766 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:53.431803 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:53.431852 (+ 49us) server_negotiation.cc:407] Connection header received
0808 14:32:53.432768 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:53.432793 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:53.433330 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:53.433740 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:53.434227 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.435896 (+ 1669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.436967 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.437756 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.437931 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:53.438926 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:53.438933 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:53.438950 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:32:53.439523 (+ 573us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:53.439594 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:32:53.439621 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:53.439722 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:53.439862 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 14:32:53.440165 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":130,"threads_started":1}
I20260808 14:32:53.442138 27370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:53.443179 27370 log.cc:824] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:53.445289 27370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5: No bootstrap required, opened a new log
I20260808 14:32:53.448796 27370 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb3d656378c47bd83852cc9759f30e5" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36381 } }
I20260808 14:32:53.449020 27370 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:53.449059 27370 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bb3d656378c47bd83852cc9759f30e5, State: Initialized, Role: FOLLOWER
I20260808 14:32:53.449487 27370 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb3d656378c47bd83852cc9759f30e5" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36381 } }
I20260808 14:32:53.449604 27370 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:32:53.449689 27370 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:32:53.449806 27370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:53.450752 27370 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb3d656378c47bd83852cc9759f30e5" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36381 } }
I20260808 14:32:53.451138 27370 leader_election.cc:302] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4bb3d656378c47bd83852cc9759f30e5; no voters:
I20260808 14:32:53.451454 27370 leader_election.cc:288] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:32:53.451598 27375 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:32:53.451872 27375 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5 [term 1 LEADER]: Becoming Leader. State: Replica: 4bb3d656378c47bd83852cc9759f30e5, State: Running, Role: LEADER
I20260808 14:32:53.452219 27375 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb3d656378c47bd83852cc9759f30e5" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36381 } }
I20260808 14:32:53.452729 27370 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:32:53.454077 27377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bb3d656378c47bd83852cc9759f30e5. Latest consensus state: current_term: 1 leader_uuid: "4bb3d656378c47bd83852cc9759f30e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb3d656378c47bd83852cc9759f30e5" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36381 } } }
I20260808 14:32:53.454213 27377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5 [sys.catalog]: This master's current role is: LEADER
I20260808 14:32:53.454506 27376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4bb3d656378c47bd83852cc9759f30e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb3d656378c47bd83852cc9759f30e5" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36381 } } }
I20260808 14:32:53.454622 27376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5 [sys.catalog]: This master's current role is: LEADER
I20260808 14:32:53.454578 27384 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:32:53.458045 27384 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:32:53.464079 27384 catalog_manager.cc:1347] Generated new cluster ID: d59f3f66de32491c8123fb46218fea4d
I20260808 14:32:53.464159 27384 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:32:53.474476 27384 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:32:53.475454 27384 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:32:53.483031 27384 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5: Generated new TSK 0
I20260808 14:32:53.483870 27384 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:32:53.491905 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:36381
--builtin_ntp_servers=127.7.125.84:41517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 14:32:53.608062 27394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:53.608357 27394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:53.608428 27394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:53.612699 27394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:53.612783 27394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:53.612874 27394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:32:53.617898 27394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:41517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:36381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:53.619225 27394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:53.620719 27394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:53.630851 27402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:53.630854 27399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:53.630854 27400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:53.630915 27394 server_base.cc:1034] running on GCE node
I20260808 14:32:53.631589 27394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:53.632210 27394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:53.633381 27394 hybrid_clock.cc:648] HybridClock initialized: now 1786199573633361 us; error 32 us; skew 500 ppm
I20260808 14:32:53.635355 27394 webserver.cc:460] Webserver started at http://127.7.125.65:44717/ using document root <none> and password file <none>
I20260808 14:32:53.635955 27394 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:53.636018 27394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:53.636266 27394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:53.638343 27394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "d7c430f098b74855a43d2fa37a3db1ca"
format_stamp: "Formatted at 2026-08-08 14:32:53 on dist-test-slave-w206"
server_key: "e425575fd85d257e7cec8f4abb859288"
server_key_iv: "09d6bbc443375b65251a9ae707749f62"
server_key_version: "encryptionkey@0"
I20260808 14:32:53.638932 27394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "d7c430f098b74855a43d2fa37a3db1ca"
format_stamp: "Formatted at 2026-08-08 14:32:53 on dist-test-slave-w206"
server_key: "e425575fd85d257e7cec8f4abb859288"
server_key_iv: "09d6bbc443375b65251a9ae707749f62"
server_key_version: "encryptionkey@0"
I20260808 14:32:53.642611 27394 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 14:32:53.645116 27411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:53.646384 27394 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 14:32:53.646518 27394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "d7c430f098b74855a43d2fa37a3db1ca"
format_stamp: "Formatted at 2026-08-08 14:32:53 on dist-test-slave-w206"
server_key: "e425575fd85d257e7cec8f4abb859288"
server_key_iv: "09d6bbc443375b65251a9ae707749f62"
server_key_version: "encryptionkey@0"
I20260808 14:32:53.646639 27394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:53.656306 27394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:53.656988 27394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:53.657186 27394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:53.657756 27394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:53.658710 27394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:32:53.658779 27394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:53.658846 27394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:32:53.658900 27394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:53.668093 27394 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:34475
I20260808 14:32:53.668116 27537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:34475 every 8 connection(s)
I20260808 14:32:53.669027 27394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:32:53.669662 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 27394
I20260808 14:32:53.669806 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:32:53.670135 7669 external_mini_cluster.cc:1428] Setting key ce0f7d75f2770f5456c6a56091afb8a2
I20260808 14:32:53.672395 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:36381
--builtin_ntp_servers=127.7.125.84:41517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 14:32:53.680580 27373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:53.671069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:47609 (local address 127.7.125.126:36381)
0808 14:32:53.671194 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:53.671198 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:53.672539 (+ 1341us) server_negotiation.cc:407] Connection header received
0808 14:32:53.674638 (+ 2099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:53.674643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:53.674712 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:53.674813 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:53.675883 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.676419 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.678314 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.678530 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.678610 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:53.680055 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:53.680058 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:53.680060 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:32:53.680074 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:53.680093 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:32:53.680101 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:53.680161 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:53.680384 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 14:32:53.680461 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 14:32:53.681998 27541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:53.671600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36381 (local address 127.7.125.65:47609)
0808 14:32:53.672360 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:53.672418 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:53.674388 (+ 1970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:53.674967 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:53.675031 (+ 64us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:53.675120 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:53.675722 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:53.675744 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.676555 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.676562 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:53.678174 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:53.678191 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.678702 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.678711 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:53.678966 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:53.679702 (+ 736us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:32:53.679707 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:53.679771 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:32:53.679869 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:53.680194 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:53.680208 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:53.680225 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 14:32:53.680369 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 14:32:53.680826 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":104,"threads_started":1}
I20260808 14:32:53.683393 27538 heartbeater.cc:347] Connected to a master server at 127.7.125.126:36381
I20260808 14:32:53.683667 27538 heartbeater.cc:464] Registering TS with master...
I20260808 14:32:53.684387 27538 heartbeater.cc:511] Master 127.7.125.126:36381 requested a full tablet report, sending...
I20260808 14:32:53.685985 27332 ts_manager.cc:194] Registered new tserver with Master: d7c430f098b74855a43d2fa37a3db1ca (127.7.125.65:34475)
I20260808 14:32:53.687265 27332 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.125.65:47609
W20260808 14:32:53.804474 27542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:53.804759 27542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:53.804801 27542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:53.809062 27542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:53.809123 27542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:53.809217 27542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:32:53.815855 27542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:41517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:36381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:53.817082 27542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:53.818563 27542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:53.828714 27547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:53.828683 27550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:53.828687 27548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:53.829104 27542 server_base.cc:1034] running on GCE node
I20260808 14:32:53.829528 27542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:53.830228 27542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:53.831418 27542 hybrid_clock.cc:648] HybridClock initialized: now 1786199573831403 us; error 26 us; skew 500 ppm
I20260808 14:32:53.837774 27542 webserver.cc:460] Webserver started at http://127.7.125.66:37635/ using document root <none> and password file <none>
I20260808 14:32:53.838413 27542 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:53.838522 27542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:53.838764 27542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:53.840965 27542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "2d5c95e4b1254e849dca375734963df5"
format_stamp: "Formatted at 2026-08-08 14:32:53 on dist-test-slave-w206"
server_key: "9c37b4f17d31374d5cc15c0dc260eb09"
server_key_iv: "476d06a6e64528287a05d6bafc676510"
server_key_version: "encryptionkey@0"
I20260808 14:32:53.841550 27542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "2d5c95e4b1254e849dca375734963df5"
format_stamp: "Formatted at 2026-08-08 14:32:53 on dist-test-slave-w206"
server_key: "9c37b4f17d31374d5cc15c0dc260eb09"
server_key_iv: "476d06a6e64528287a05d6bafc676510"
server_key_version: "encryptionkey@0"
I20260808 14:32:53.845398 27542 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:32:53.847913 27558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:53.849104 27542 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:32:53.849241 27542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "2d5c95e4b1254e849dca375734963df5"
format_stamp: "Formatted at 2026-08-08 14:32:53 on dist-test-slave-w206"
server_key: "9c37b4f17d31374d5cc15c0dc260eb09"
server_key_iv: "476d06a6e64528287a05d6bafc676510"
server_key_version: "encryptionkey@0"
I20260808 14:32:53.849367 27542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:53.876001 27542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:53.876792 27542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:53.877020 27542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:53.877673 27542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:53.878683 27542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:32:53.878758 27542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:53.878834 27542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:32:53.878882 27542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:53.889081 27542 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:46833
I20260808 14:32:53.889104 27681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:46833 every 8 connection(s)
I20260808 14:32:53.890026 27542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:32:53.891316 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 27542
I20260808 14:32:53.891443 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:32:53.891779 7669 external_mini_cluster.cc:1428] Setting key b61d9edb571b1d6776eb7627e84ac123
I20260808 14:32:53.894177 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:36381
--builtin_ntp_servers=127.7.125.84:41517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 14:32:53.901746 27373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:53.892180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:34699 (local address 127.7.125.126:36381)
0808 14:32:53.892365 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:53.892369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:53.893652 (+ 1283us) server_negotiation.cc:407] Connection header received
0808 14:32:53.895719 (+ 2067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:53.895721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:53.895767 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:53.895825 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:53.896865 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.897637 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.899588 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.899798 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.899884 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:53.901275 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:53.901278 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:53.901280 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:32:53.901292 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:53.901312 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:32:53.901320 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:53.901384 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:53.901573 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 14:32:53.901629 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 14:32:53.903133 27684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:53.892722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36381 (local address 127.7.125.66:34699)
0808 14:32:53.893465 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:53.893525 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:53.895478 (+ 1953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:53.896027 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:53.896039 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:53.896164 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:53.896704 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:53.896717 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.897780 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.897786 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:53.899459 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:53.899474 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:53.899978 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:53.899985 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:53.900247 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:53.900938 (+ 691us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:32:53.900942 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:53.901002 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:32:53.901096 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:53.901408 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:53.901420 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:53.901428 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 14:32:53.901572 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 14:32:53.901916 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":151,"threads_started":1}
I20260808 14:32:53.904318 27682 heartbeater.cc:347] Connected to a master server at 127.7.125.126:36381
I20260808 14:32:53.904596 27682 heartbeater.cc:464] Registering TS with master...
I20260808 14:32:53.905133 27682 heartbeater.cc:511] Master 127.7.125.126:36381 requested a full tablet report, sending...
I20260808 14:32:53.906181 27332 ts_manager.cc:194] Registered new tserver with Master: 2d5c95e4b1254e849dca375734963df5 (127.7.125.66:46833)
I20260808 14:32:53.906765 27332 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.125.66:34699
W20260808 14:32:54.009889 27685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:54.010150 27685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:54.010187 27685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:54.014489 27685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:54.014550 27685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:54.014609 27685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:32:54.019372 27685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:41517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:36381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:54.020490 27685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:54.021896 27685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:54.031684 27690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:54.031801 27693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:54.031777 27685 server_base.cc:1034] running on GCE node
W20260808 14:32:54.031785 27691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:54.032307 27685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:54.032967 27685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:54.034165 27685 hybrid_clock.cc:648] HybridClock initialized: now 1786199574034147 us; error 33 us; skew 500 ppm
I20260808 14:32:54.036207 27685 webserver.cc:460] Webserver started at http://127.7.125.67:41791/ using document root <none> and password file <none>
I20260808 14:32:54.036830 27685 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:54.036929 27685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:54.037168 27685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:54.039389 27685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "505ab037ac6844e4a7107758849712d8"
format_stamp: "Formatted at 2026-08-08 14:32:54 on dist-test-slave-w206"
server_key: "46a6dce61fb6646e3cb8cd8c09a016df"
server_key_iv: "22b3e8ecfed9625d3b083b52041bef4f"
server_key_version: "encryptionkey@0"
I20260808 14:32:54.039991 27685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "505ab037ac6844e4a7107758849712d8"
format_stamp: "Formatted at 2026-08-08 14:32:54 on dist-test-slave-w206"
server_key: "46a6dce61fb6646e3cb8cd8c09a016df"
server_key_iv: "22b3e8ecfed9625d3b083b52041bef4f"
server_key_version: "encryptionkey@0"
I20260808 14:32:54.043985 27685 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 14:32:54.046592 27699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:54.047942 27685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 14:32:54.048090 27685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "505ab037ac6844e4a7107758849712d8"
format_stamp: "Formatted at 2026-08-08 14:32:54 on dist-test-slave-w206"
server_key: "46a6dce61fb6646e3cb8cd8c09a016df"
server_key_iv: "22b3e8ecfed9625d3b083b52041bef4f"
server_key_version: "encryptionkey@0"
I20260808 14:32:54.048218 27685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:54.093119 27685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:54.093887 27685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:54.094112 27685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:54.094753 27685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:54.095741 27685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:32:54.095813 27685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:54.095886 27685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:32:54.095938 27685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:54.105113 27685 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:38815
I20260808 14:32:54.105130 27819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:38815 every 8 connection(s)
I20260808 14:32:54.106036 27685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:32:54.113191 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 27685
I20260808 14:32:54.113336 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:32:54.113633 7669 external_mini_cluster.cc:1428] Setting key 6c8cf6cc359c4e441692e7a6238a3cf5
I20260808 14:32:54.116040 27373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:54.108006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:53141 (local address 127.7.125.126:36381)
0808 14:32:54.108186 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:54.108191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:54.109254 (+ 1063us) server_negotiation.cc:407] Connection header received
0808 14:32:54.110700 (+ 1446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:54.110703 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:54.110758 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:54.110844 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:54.111933 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.112473 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.113732 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.113974 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.114095 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:54.115457 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:54.115460 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:54.115462 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:32:54.115473 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:54.115493 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:32:54.115501 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:54.115582 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:54.115802 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 14:32:54.115873 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 14:32:54.116916 27823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:54.108405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36381 (local address 127.7.125.67:53141)
0808 14:32:54.109084 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:54.109126 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:54.110469 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:54.111082 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:54.111089 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:54.111201 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:54.111743 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:54.111756 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.112615 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.112618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:54.113597 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:54.113606 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.114141 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.114145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:54.114365 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:54.115135 (+ 770us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:32:54.115137 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:54.115192 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:32:54.115275 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:54.115608 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:54.115617 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:54.115623 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 14:32:54.115770 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 14:32:54.116071 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":121,"threads_started":1}
I20260808 14:32:54.118052 27820 heartbeater.cc:347] Connected to a master server at 127.7.125.126:36381
I20260808 14:32:54.118358 27820 heartbeater.cc:464] Registering TS with master...
I20260808 14:32:54.118932 27820 heartbeater.cc:511] Master 127.7.125.126:36381 requested a full tablet report, sending...
I20260808 14:32:54.120035 27332 ts_manager.cc:194] Registered new tserver with Master: 505ab037ac6844e4a7107758849712d8 (127.7.125.67:38815)
I20260808 14:32:54.120568 27332 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.125.67:53141
I20260808 14:32:54.128224 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:32:54.134650 27373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:54.130494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34962 (local address 127.7.125.126:36381)
0808 14:32:54.130649 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:54.130655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:54.130862 (+ 207us) server_negotiation.cc:407] Connection header received
0808 14:32:54.131040 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:54.131043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:54.131095 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:54.131183 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:54.131515 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.132038 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.132901 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.133125 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.133247 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:54.134121 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:54.134125 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:54.134127 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:32:54.134139 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:54.134158 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:32:54.134166 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:54.134310 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:54.134461 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 14:32:54.134500 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:32:54.137985 27332 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:32:54.141355 27332 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:32:54.158217 27832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:54.150894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:46833 (local address 127.0.0.1:55690)
0808 14:32:54.152605 (+ 1711us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:54.152616 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:54.152737 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:54.153265 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:54.153271 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:54.153297 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:54.153597 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:54.153607 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.154817 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.154821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:54.156397 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:54.156409 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.157586 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.157590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:54.157724 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:54.157772 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 14:32:54.157900 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 14:32:54.158037 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1656,"mutex_wait_us":1202,"spinlock_wait_cycles":1280,"thread_start_us":90,"threads_started":1}
I20260808 14:32:54.158712 27837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:54.151259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:34475 (local address 127.0.0.1:58332)
0808 14:32:54.152457 (+ 1198us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:54.152469 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:54.152542 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:54.153619 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:54.153622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:54.153637 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:54.154697 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:54.154717 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.155978 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.155984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:54.157019 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:54.157030 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.158269 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.158276 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:54.158377 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:54.158404 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 14:32:54.158477 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 14:32:54.158542 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1134,"mutex_wait_us":939,"thread_start_us":132,"threads_started":1}
I20260808 14:32:54.158730 27835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:54.151510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55690 (local address 127.7.125.66:46833)
0808 14:32:54.151913 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:54.151925 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:54.152709 (+ 784us) server_negotiation.cc:407] Connection header received
0808 14:32:54.152852 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:54.152858 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:54.153086 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:54.153320 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:54.153738 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.154690 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.156538 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.157441 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.157530 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:54.157676 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:54.158435 (+ 759us) server_negotiation.cc:299] Negotiation successful
0808 14:32:54.158552 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":163,"threads_started":1}
I20260808 14:32:54.159264 27834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:54.151420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58332 (local address 127.7.125.65:34475)
0808 14:32:54.153046 (+ 1626us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:54.153052 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:54.153074 (+ 22us) server_negotiation.cc:407] Connection header received
0808 14:32:54.153186 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:54.153193 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:54.153444 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:54.153636 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:54.154844 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.155861 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.157165 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.157868 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.157963 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:54.158149 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:54.158962 (+ 813us) server_negotiation.cc:299] Negotiation successful
0808 14:32:54.159099 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1517,"thread_start_us":106,"threads_started":1}
I20260808 14:32:54.159389 27833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:54.150894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:38815 (local address 127.0.0.1:40670)
0808 14:32:54.152094 (+ 1200us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:54.152186 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:54.152387 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:54.153333 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:54.153336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:54.153364 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:54.153770 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:54.153780 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.155223 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.155226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:54.156422 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:54.156429 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.158929 (+ 2500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.158934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:54.159077 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:54.159100 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 14:32:54.159147 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 14:32:54.159218 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1004,"mutex_wait_us":223,"spinlock_wait_cycles":1408,"thread_start_us":100,"threads_started":1}
I20260808 14:32:54.161532 27466 tablet_service.cc:1467] Processing CreateTablet for tablet 96a25d8e8b9b49bb9d10f18db81daf36 (DEFAULT_TABLE table=test-table [id=35f19f0225f8489c910d3c95ed7b85d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:32:54.161471 27609 tablet_service.cc:1467] Processing CreateTablet for tablet 96a25d8e8b9b49bb9d10f18db81daf36 (DEFAULT_TABLE table=test-table [id=35f19f0225f8489c910d3c95ed7b85d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:32:54.162566 27609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96a25d8e8b9b49bb9d10f18db81daf36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:54.162568 27466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96a25d8e8b9b49bb9d10f18db81daf36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:54.162768 27836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:54.151662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40670 (local address 127.7.125.67:38815)
0808 14:32:54.152736 (+ 1074us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:54.152750 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:54.152778 (+ 28us) server_negotiation.cc:407] Connection header received
0808 14:32:54.152852 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:54.152858 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:54.153141 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:54.153552 (+ 411us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:54.153910 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.155079 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.157728 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.158781 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.162146 (+ 3365us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:54.162389 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:54.162449 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 14:32:54.162551 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":944,"thread_start_us":178,"threads_started":1}
I20260808 14:32:54.165896 27750 tablet_service.cc:1467] Processing CreateTablet for tablet 96a25d8e8b9b49bb9d10f18db81daf36 (DEFAULT_TABLE table=test-table [id=35f19f0225f8489c910d3c95ed7b85d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:32:54.167088 27750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96a25d8e8b9b49bb9d10f18db81daf36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:54.169404 27838 tablet_bootstrap.cc:492] T 96a25d8e8b9b49bb9d10f18db81daf36 P 2d5c95e4b1254e849dca375734963df5: Bootstrap starting.
I20260808 14:32:54.170894 27839 tablet_bootstrap.cc:492] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca: Bootstrap starting.
I20260808 14:32:54.171793 27838 tablet_bootstrap.cc:654] T 96a25d8e8b9b49bb9d10f18db81daf36 P 2d5c95e4b1254e849dca375734963df5: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:54.172533 27838 log.cc:824] T 96a25d8e8b9b49bb9d10f18db81daf36 P 2d5c95e4b1254e849dca375734963df5: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:54.173131 27839 tablet_bootstrap.cc:654] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:54.174000 27839 log.cc:824] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:54.175118 27841 tablet_bootstrap.cc:492] T 96a25d8e8b9b49bb9d10f18db81daf36 P 505ab037ac6844e4a7107758849712d8: Bootstrap starting.
I20260808 14:32:54.175583 27838 tablet_bootstrap.cc:492] T 96a25d8e8b9b49bb9d10f18db81daf36 P 2d5c95e4b1254e849dca375734963df5: No bootstrap required, opened a new log
I20260808 14:32:54.175798 27838 ts_tablet_manager.cc:1397] T 96a25d8e8b9b49bb9d10f18db81daf36 P 2d5c95e4b1254e849dca375734963df5: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.002s
I20260808 14:32:54.176160 27839 tablet_bootstrap.cc:492] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca: No bootstrap required, opened a new log
I20260808 14:32:54.176357 27839 ts_tablet_manager.cc:1397] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 14:32:54.177420 27841 tablet_bootstrap.cc:654] T 96a25d8e8b9b49bb9d10f18db81daf36 P 505ab037ac6844e4a7107758849712d8: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:54.178279 27841 log.cc:824] T 96a25d8e8b9b49bb9d10f18db81daf36 P 505ab037ac6844e4a7107758849712d8: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:54.179833 27839 raft_consensus.cc:348] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505ab037ac6844e4a7107758849712d8" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 38815 } } peers { permanent_uuid: "2d5c95e4b1254e849dca375734963df5" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 46833 } } peers { permanent_uuid: "d7c430f098b74855a43d2fa37a3db1ca" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34475 } }
I20260808 14:32:54.179863 27838 raft_consensus.cc:348] T 96a25d8e8b9b49bb9d10f18db81daf36 P 2d5c95e4b1254e849dca375734963df5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505ab037ac6844e4a7107758849712d8" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 38815 } } peers { permanent_uuid: "2d5c95e4b1254e849dca375734963df5" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 46833 } } peers { permanent_uuid: "d7c430f098b74855a43d2fa37a3db1ca" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34475 } }
I20260808 14:32:54.180104 27839 raft_consensus.cc:374] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:54.180131 27838 raft_consensus.cc:374] T 96a25d8e8b9b49bb9d10f18db81daf36 P 2d5c95e4b1254e849dca375734963df5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:54.180159 27839 raft_consensus.cc:729] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7c430f098b74855a43d2fa37a3db1ca, State: Initialized, Role: FOLLOWER
I20260808 14:32:54.180191 27838 raft_consensus.cc:729] T 96a25d8e8b9b49bb9d10f18db81daf36 P 2d5c95e4b1254e849dca375734963df5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d5c95e4b1254e849dca375734963df5, State: Initialized, Role: FOLLOWER
I20260808 14:32:54.180540 27841 tablet_bootstrap.cc:492] T 96a25d8e8b9b49bb9d10f18db81daf36 P 505ab037ac6844e4a7107758849712d8: No bootstrap required, opened a new log
I20260808 14:32:54.180722 27841 ts_tablet_manager.cc:1397] T 96a25d8e8b9b49bb9d10f18db81daf36 P 505ab037ac6844e4a7107758849712d8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:32:54.180637 27839 consensus_queue.cc:260] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505ab037ac6844e4a7107758849712d8" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 38815 } } peers { permanent_uuid: "2d5c95e4b1254e849dca375734963df5" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 46833 } } peers { permanent_uuid: "d7c430f098b74855a43d2fa37a3db1ca" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34475 } }
I20260808 14:32:54.180661 27838 consensus_queue.cc:260] T 96a25d8e8b9b49bb9d10f18db81daf36 P 2d5c95e4b1254e849dca375734963df5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505ab037ac6844e4a7107758849712d8" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 38815 } } peers { permanent_uuid: "2d5c95e4b1254e849dca375734963df5" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 46833 } } peers { permanent_uuid: "d7c430f098b74855a43d2fa37a3db1ca" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34475 } }
I20260808 14:32:54.181691 27838 ts_tablet_manager.cc:1428] T 96a25d8e8b9b49bb9d10f18db81daf36 P 2d5c95e4b1254e849dca375734963df5: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260808 14:32:54.181708 27839 ts_tablet_manager.cc:1428] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260808 14:32:54.182072 27538 heartbeater.cc:503] Master 127.7.125.126:36381 was elected leader, sending a full tablet report...
I20260808 14:32:54.183259 27682 heartbeater.cc:503] Master 127.7.125.126:36381 was elected leader, sending a full tablet report...
I20260808 14:32:54.184676 27841 raft_consensus.cc:348] T 96a25d8e8b9b49bb9d10f18db81daf36 P 505ab037ac6844e4a7107758849712d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505ab037ac6844e4a7107758849712d8" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 38815 } } peers { permanent_uuid: "2d5c95e4b1254e849dca375734963df5" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 46833 } } peers { permanent_uuid: "d7c430f098b74855a43d2fa37a3db1ca" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34475 } }
I20260808 14:32:54.184888 27841 raft_consensus.cc:374] T 96a25d8e8b9b49bb9d10f18db81daf36 P 505ab037ac6844e4a7107758849712d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:54.184954 27841 raft_consensus.cc:729] T 96a25d8e8b9b49bb9d10f18db81daf36 P 505ab037ac6844e4a7107758849712d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 505ab037ac6844e4a7107758849712d8, State: Initialized, Role: FOLLOWER
I20260808 14:32:54.185439 27841 consensus_queue.cc:260] T 96a25d8e8b9b49bb9d10f18db81daf36 P 505ab037ac6844e4a7107758849712d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505ab037ac6844e4a7107758849712d8" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 38815 } } peers { permanent_uuid: "2d5c95e4b1254e849dca375734963df5" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 46833 } } peers { permanent_uuid: "d7c430f098b74855a43d2fa37a3db1ca" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34475 } }
I20260808 14:32:54.186120 27820 heartbeater.cc:503] Master 127.7.125.126:36381 was elected leader, sending a full tablet report...
I20260808 14:32:54.186530 27841 ts_tablet_manager.cc:1428] T 96a25d8e8b9b49bb9d10f18db81daf36 P 505ab037ac6844e4a7107758849712d8: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 14:32:54.292182 27844 raft_consensus.cc:482] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:32:54.292495 27844 raft_consensus.cc:504] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505ab037ac6844e4a7107758849712d8" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 38815 } } peers { permanent_uuid: "2d5c95e4b1254e849dca375734963df5" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 46833 } } peers { permanent_uuid: "d7c430f098b74855a43d2fa37a3db1ca" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34475 } }
I20260808 14:32:54.293751 27844 leader_election.cc:288] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 505ab037ac6844e4a7107758849712d8 (127.7.125.67:38815), 2d5c95e4b1254e849dca375734963df5 (127.7.125.66:46833)
I20260808 14:32:54.298091 27836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:54.294173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:56821 (local address 127.7.125.67:38815)
0808 14:32:54.294328 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:54.294333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:54.294571 (+ 238us) server_negotiation.cc:407] Connection header received
0808 14:32:54.294828 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:54.294831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:54.294914 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:54.295058 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:54.295532 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.296018 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.297106 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.297523 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.297650 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:54.297700 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:54.297882 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 14:32:54.297952 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 14:32:54.298120 27835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:54.294154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:54317 (local address 127.7.125.66:46833)
0808 14:32:54.294298 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:54.294303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:54.295037 (+ 734us) server_negotiation.cc:407] Connection header received
0808 14:32:54.295139 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:54.295142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:54.295208 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:54.295341 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:54.295635 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.296121 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.297171 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.297557 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.297689 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:54.297738 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:54.297925 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 14:32:54.297999 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 14:32:54.298134 27848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:54.294061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:38815 (local address 127.7.125.65:56821)
0808 14:32:54.294458 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:54.294470 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:54.294616 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:54.295133 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:54.295139 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:54.295154 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:54.295378 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:54.295385 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.296132 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.296136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:54.296964 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:54.296973 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.297647 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.297651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:54.297759 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:54.297777 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 14:32:54.297867 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 14:32:54.297935 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":123,"threads_started":1}
I20260808 14:32:54.298137 27849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:54.294620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:46833 (local address 127.7.125.65:54317)
0808 14:32:54.294919 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:54.294931 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:54.295048 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:54.295352 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:54.295356 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:54.295374 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:54.295508 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:54.295513 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.296246 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.296252 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:54.297035 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:54.297044 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.297683 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.297687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:54.297789 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:54.297817 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 14:32:54.297906 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 14:32:54.297973 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":113,"threads_started":1}
I20260808 14:32:54.298849 27772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96a25d8e8b9b49bb9d10f18db81daf36" candidate_uuid: "d7c430f098b74855a43d2fa37a3db1ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "505ab037ac6844e4a7107758849712d8" is_pre_election: true
I20260808 14:32:54.298854 27632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96a25d8e8b9b49bb9d10f18db81daf36" candidate_uuid: "d7c430f098b74855a43d2fa37a3db1ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d5c95e4b1254e849dca375734963df5" is_pre_election: true
I20260808 14:32:54.299189 27772 raft_consensus.cc:2454] T 96a25d8e8b9b49bb9d10f18db81daf36 P 505ab037ac6844e4a7107758849712d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7c430f098b74855a43d2fa37a3db1ca in term 0.
I20260808 14:32:54.299208 27632 raft_consensus.cc:2454] T 96a25d8e8b9b49bb9d10f18db81daf36 P 2d5c95e4b1254e849dca375734963df5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7c430f098b74855a43d2fa37a3db1ca in term 0.
I20260808 14:32:54.299695 27414 leader_election.cc:302] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d5c95e4b1254e849dca375734963df5, 505ab037ac6844e4a7107758849712d8, d7c430f098b74855a43d2fa37a3db1ca; no voters:
I20260808 14:32:54.300014 27844 raft_consensus.cc:2781] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:32:54.300115 27844 raft_consensus.cc:482] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:32:54.300174 27844 raft_consensus.cc:3037] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:54.301254 27844 raft_consensus.cc:504] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505ab037ac6844e4a7107758849712d8" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 38815 } } peers { permanent_uuid: "2d5c95e4b1254e849dca375734963df5" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 46833 } } peers { permanent_uuid: "d7c430f098b74855a43d2fa37a3db1ca" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34475 } }
I20260808 14:32:54.301640 27844 leader_election.cc:288] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca [CANDIDATE]: Term 1 election: Requested vote from peers 505ab037ac6844e4a7107758849712d8 (127.7.125.67:38815), 2d5c95e4b1254e849dca375734963df5 (127.7.125.66:46833)
I20260808 14:32:54.301995 27632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96a25d8e8b9b49bb9d10f18db81daf36" candidate_uuid: "d7c430f098b74855a43d2fa37a3db1ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d5c95e4b1254e849dca375734963df5"
I20260808 14:32:54.302038 27772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96a25d8e8b9b49bb9d10f18db81daf36" candidate_uuid: "d7c430f098b74855a43d2fa37a3db1ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "505ab037ac6844e4a7107758849712d8"
I20260808 14:32:54.302142 27632 raft_consensus.cc:3037] T 96a25d8e8b9b49bb9d10f18db81daf36 P 2d5c95e4b1254e849dca375734963df5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:54.302160 27772 raft_consensus.cc:3037] T 96a25d8e8b9b49bb9d10f18db81daf36 P 505ab037ac6844e4a7107758849712d8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:54.303344 27632 raft_consensus.cc:2454] T 96a25d8e8b9b49bb9d10f18db81daf36 P 2d5c95e4b1254e849dca375734963df5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7c430f098b74855a43d2fa37a3db1ca in term 1.
I20260808 14:32:54.303403 27772 raft_consensus.cc:2454] T 96a25d8e8b9b49bb9d10f18db81daf36 P 505ab037ac6844e4a7107758849712d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7c430f098b74855a43d2fa37a3db1ca in term 1.
I20260808 14:32:54.303756 27414 leader_election.cc:302] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 505ab037ac6844e4a7107758849712d8, d7c430f098b74855a43d2fa37a3db1ca; no voters:
I20260808 14:32:54.303946 27844 raft_consensus.cc:2781] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:32:54.304230 27844 raft_consensus.cc:686] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca [term 1 LEADER]: Becoming Leader. State: Replica: d7c430f098b74855a43d2fa37a3db1ca, State: Running, Role: LEADER
I20260808 14:32:54.304586 27844 consensus_queue.cc:237] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505ab037ac6844e4a7107758849712d8" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 38815 } } peers { permanent_uuid: "2d5c95e4b1254e849dca375734963df5" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 46833 } } peers { permanent_uuid: "d7c430f098b74855a43d2fa37a3db1ca" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34475 } }
I20260808 14:32:54.307847 27332 catalog_manager.cc:5495] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca reported cstate change: term changed from 0 to 1, leader changed from <none> to d7c430f098b74855a43d2fa37a3db1ca (127.7.125.65). New cstate: current_term: 1 leader_uuid: "d7c430f098b74855a43d2fa37a3db1ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505ab037ac6844e4a7107758849712d8" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 38815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d5c95e4b1254e849dca375734963df5" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 46833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7c430f098b74855a43d2fa37a3db1ca" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34475 } health_report { overall_health: HEALTHY } } }
I20260808 14:32:54.349694 27834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:54.346057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58342 (local address 127.7.125.65:34475)
0808 14:32:54.346208 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:54.346213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:54.346229 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:32:54.346304 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:54.346306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:54.346364 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:54.346452 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:32:54.346775 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.347308 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.348206 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:54.348420 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:54.348526 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:54.348850 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:32:54.349249 (+ 399us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:32:54.349401 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:54.349510 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 14:32:54.349569 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 14:32:54.356338 27772 raft_consensus.cc:1261] T 96a25d8e8b9b49bb9d10f18db81daf36 P 505ab037ac6844e4a7107758849712d8 [term 1 FOLLOWER]: Refusing update from remote peer d7c430f098b74855a43d2fa37a3db1ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:32:54.356338 27632 raft_consensus.cc:1261] T 96a25d8e8b9b49bb9d10f18db81daf36 P 2d5c95e4b1254e849dca375734963df5 [term 1 FOLLOWER]: Refusing update from remote peer d7c430f098b74855a43d2fa37a3db1ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260808 14:32:54.356935 27821 tablet.cc:2369] T 96a25d8e8b9b49bb9d10f18db81daf36 P 505ab037ac6844e4a7107758849712d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:32:54.357048 27844 consensus_queue.cc:1035] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d5c95e4b1254e849dca375734963df5" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 46833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:32:54.357272 27850 consensus_queue.cc:1035] T 96a25d8e8b9b49bb9d10f18db81daf36 P d7c430f098b74855a43d2fa37a3db1ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "505ab037ac6844e4a7107758849712d8" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 38815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:32:54.365393 27854 mvcc.cc:205] Tried to move back new op lower bound from 7316273456555900928 to 7316273456354672640. Current Snapshot: MvccSnapshot[applied={T|T < 7316273456555900928}]
I20260808 14:32:54.374850 27331 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:32:54.375077 27331 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:32:54.377903 27331 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4bb3d656378c47bd83852cc9759f30e5: Sending DeleteTablet for 3 replicas of tablet 96a25d8e8b9b49bb9d10f18db81daf36
I20260808 14:32:54.378801 27609 tablet_service.cc:1514] Processing DeleteTablet for tablet 96a25d8e8b9b49bb9d10f18db81daf36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:32:54 UTC) from {username='slave'} at 127.0.0.1:55690
I20260808 14:32:54.378782 27750 tablet_service.cc:1514] Processing DeleteTablet for tablet 96a25d8e8b9b49bb9d10f18db81daf36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:32:54 UTC) from {username='slave'} at 127.0.0.1:40670
I20260808 14:32:54.379355 27466 tablet_service.cc:1514] Processing DeleteTablet for tablet 96a25d8e8b9b49bb9d10f18db81daf36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:32:54 UTC) from {username='slave'} at 127.0.0.1:58332
I20260808 14:32:54.379725 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 27394
I20260808 14:32:54.382299 27862 tablet_replica.cc:330] T 96a25d8e8b9b49bb9d10f18db81daf36 P 505ab037ac6844e4a7107758849712d8: stopping tablet replica
I20260808 14:32:54.382496 27863 tablet_replica.cc:330] T 96a25d8e8b9b49bb9d10f18db81daf36 P 2d5c95e4b1254e849dca375734963df5: stopping tablet replica
I20260808 14:32:54.382645 27862 raft_consensus.cc:2229] T 96a25d8e8b9b49bb9d10f18db81daf36 P 505ab037ac6844e4a7107758849712d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:32:54.382854 27863 raft_consensus.cc:2229] T 96a25d8e8b9b49bb9d10f18db81daf36 P 2d5c95e4b1254e849dca375734963df5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:32:54.383010 27862 raft_consensus.cc:2258] T 96a25d8e8b9b49bb9d10f18db81daf36 P 505ab037ac6844e4a7107758849712d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:32:54.383188 27863 raft_consensus.cc:2258] T 96a25d8e8b9b49bb9d10f18db81daf36 P 2d5c95e4b1254e849dca375734963df5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:32:54.384478 27863 ts_tablet_manager.cc:1905] T 96a25d8e8b9b49bb9d10f18db81daf36 P 2d5c95e4b1254e849dca375734963df5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:32:54.384577 27862 ts_tablet_manager.cc:1905] T 96a25d8e8b9b49bb9d10f18db81daf36 P 505ab037ac6844e4a7107758849712d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:32:54.386689 27863 ts_tablet_manager.cc:1918] T 96a25d8e8b9b49bb9d10f18db81daf36 P 2d5c95e4b1254e849dca375734963df5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:32:54.386817 27863 log.cc:1196] T 96a25d8e8b9b49bb9d10f18db81daf36 P 2d5c95e4b1254e849dca375734963df5: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/96a25d8e8b9b49bb9d10f18db81daf36
I20260808 14:32:54.387207 27863 ts_tablet_manager.cc:1939] T 96a25d8e8b9b49bb9d10f18db81daf36 P 2d5c95e4b1254e849dca375734963df5: Deleting consensus metadata
I20260808 14:32:54.388026 27862 ts_tablet_manager.cc:1918] T 96a25d8e8b9b49bb9d10f18db81daf36 P 505ab037ac6844e4a7107758849712d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:32:54.388147 27862 log.cc:1196] T 96a25d8e8b9b49bb9d10f18db81daf36 P 505ab037ac6844e4a7107758849712d8: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/96a25d8e8b9b49bb9d10f18db81daf36
I20260808 14:32:54.388374 27319 catalog_manager.cc:4841] TS 2d5c95e4b1254e849dca375734963df5 (127.7.125.66:46833): tablet 96a25d8e8b9b49bb9d10f18db81daf36 (table test-table [id=35f19f0225f8489c910d3c95ed7b85d3]) successfully deleted
I20260808 14:32:54.388490 27862 ts_tablet_manager.cc:1939] T 96a25d8e8b9b49bb9d10f18db81daf36 P 505ab037ac6844e4a7107758849712d8: Deleting consensus metadata
W20260808 14:32:54.389837 27317 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.125.65:34475 (error 108)
I20260808 14:32:54.390038 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 27542
I20260808 14:32:54.392127 27318 catalog_manager.cc:4841] TS 505ab037ac6844e4a7107758849712d8 (127.7.125.67:38815): tablet 96a25d8e8b9b49bb9d10f18db81daf36 (table test-table [id=35f19f0225f8489c910d3c95ed7b85d3]) successfully deleted
I20260808 14:32:54.392274 27833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:54.391872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:34475 (local address 127.0.0.1:58358)
0808 14:32:54.392022 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:54.392105 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.65:34475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 14:32:54.392402 27317 catalog_manager.cc:4584] TS d7c430f098b74855a43d2fa37a3db1ca (127.7.125.65:34475): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 96a25d8e8b9b49bb9d10f18db81daf36: Network error: Client connection negotiation failed: client connection to 127.7.125.65:34475: connect: Connection refused (error 111)
I20260808 14:32:54.397321 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 27685
I20260808 14:32:54.404412 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 27301
2026-08-08T14:32:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1184 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:32:54 dist-test-slave-w206 krb5kdc[27869](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:32:54 dist-test-slave-w206 krb5kdc[27869](info): set up 2 sockets
Aug 08 14:32:54 dist-test-slave-w206 krb5kdc[27869](info): commencing operation
krb5kdc: starting...
W20260808 14:32:55.491705 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:32:55 dist-test-slave-w206 krb5kdc[27869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:32:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:32:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:32:55.626379 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:39481
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:41827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:39481
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:32:55.743636 27882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:55.743901 27882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:55.743943 27882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:55.747874 27882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:32:55.747931 27882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:55.747949 27882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:55.747967 27882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:32:55.747982 27882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:32:55.753189 27882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:41827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:39481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:39481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:55.754280 27882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:55.755267 27882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:55.764137 27890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:55.764112 27887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:55.764147 27888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:55.764380 27882 server_base.cc:1034] running on GCE node
I20260808 14:32:55.765148 27882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:55.766299 27882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:55.767467 27882 hybrid_clock.cc:648] HybridClock initialized: now 1786199575767438 us; error 41 us; skew 500 ppm
Aug 08 14:32:55 dist-test-slave-w206 krb5kdc[27869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199575, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:32:55.770517 27882 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:32:55.771622 27882 webserver.cc:460] Webserver started at http://127.7.125.126:38685/ using document root <none> and password file <none>
I20260808 14:32:55.772257 27882 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:55.772325 27882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:55.772574 27882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:55.774793 27882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "224126a463964115a0832507fce64f03"
format_stamp: "Formatted at 2026-08-08 14:32:55 on dist-test-slave-w206"
server_key: "b9afa75c05c9df55bc2bb4927bf7f488"
server_key_iv: "375a558704668bab0255ca24c6ece466"
server_key_version: "encryptionkey@0"
I20260808 14:32:55.775502 27882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "224126a463964115a0832507fce64f03"
format_stamp: "Formatted at 2026-08-08 14:32:55 on dist-test-slave-w206"
server_key: "b9afa75c05c9df55bc2bb4927bf7f488"
server_key_iv: "375a558704668bab0255ca24c6ece466"
server_key_version: "encryptionkey@0"
I20260808 14:32:55.779456 27882 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:32:55.782071 27897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:55.783473 27882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:32:55.783614 27882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "224126a463964115a0832507fce64f03"
format_stamp: "Formatted at 2026-08-08 14:32:55 on dist-test-slave-w206"
server_key: "b9afa75c05c9df55bc2bb4927bf7f488"
server_key_iv: "375a558704668bab0255ca24c6ece466"
server_key_version: "encryptionkey@0"
I20260808 14:32:55.783733 27882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:55.808465 27882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:55.811794 27882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:55.812038 27882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:55.819746 27882 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:39481
I20260808 14:32:55.819783 27953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:39481 every 8 connection(s)
I20260808 14:32:55.820791 27882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:32:55.823709 27955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:55.824719 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 27882
I20260808 14:32:55.824875 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:32:55.825249 7669 external_mini_cluster.cc:1428] Setting key 93858d762fe3f57f96019eb851dddea2
I20260808 14:32:55.830534 27955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03: Bootstrap starting.
I20260808 14:32:55.833400 27955 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:55.834321 27955 log.cc:824] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:32:55 dist-test-slave-w206 krb5kdc[27869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:32:55.836984 27955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03: No bootstrap required, opened a new log
I20260808 14:32:55.840963 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:55.826656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43188 (local address 127.7.125.126:39481)
0808 14:32:55.827134 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:55.827154 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:55.827208 (+ 54us) server_negotiation.cc:407] Connection header received
0808 14:32:55.827915 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:55.827933 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:55.828255 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:55.828584 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:55.829501 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:55.830480 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:55.831637 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:55.832110 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:55.832237 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:55.835284 (+ 3047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:55.835290 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:55.835309 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:55.835339 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:55.837532 (+ 2193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:55.837998 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:55.838004 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:55.838011 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:55.838156 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:55.838442 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:55.838445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:55.838449 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:55.839025 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:55.839251 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:55.839586 (+ 335us) server_negotiation.cc:299] Negotiation successful
0808 14:32:55.839937 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":154,"threads_started":1}
I20260808 14:32:55.841791 27955 raft_consensus.cc:348] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224126a463964115a0832507fce64f03" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39481 } }
I20260808 14:32:55.842080 27955 raft_consensus.cc:374] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:55.842147 27955 raft_consensus.cc:729] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 224126a463964115a0832507fce64f03, State: Initialized, Role: FOLLOWER
I20260808 14:32:55.842578 27955 consensus_queue.cc:260] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224126a463964115a0832507fce64f03" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39481 } }
I20260808 14:32:55.842698 27955 raft_consensus.cc:388] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:32:55.842768 27955 raft_consensus.cc:482] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:32:55.842869 27955 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:55.844064 27955 raft_consensus.cc:504] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224126a463964115a0832507fce64f03" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39481 } }
I20260808 14:32:55.844431 27955 leader_election.cc:302] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 224126a463964115a0832507fce64f03; no voters:
I20260808 14:32:55.844753 27955 leader_election.cc:288] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:32:55.844906 27960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:32:55.845152 27960 raft_consensus.cc:686] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03 [term 1 LEADER]: Becoming Leader. State: Replica: 224126a463964115a0832507fce64f03, State: Running, Role: LEADER
I20260808 14:32:55.845481 27960 consensus_queue.cc:237] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224126a463964115a0832507fce64f03" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39481 } }
I20260808 14:32:55.846086 27955 sys_catalog.cc:564] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:32:55.847343 27961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 224126a463964115a0832507fce64f03. Latest consensus state: current_term: 1 leader_uuid: "224126a463964115a0832507fce64f03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224126a463964115a0832507fce64f03" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39481 } } }
I20260808 14:32:55.847333 27962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "224126a463964115a0832507fce64f03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224126a463964115a0832507fce64f03" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39481 } } }
I20260808 14:32:55.847499 27961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03 [sys.catalog]: This master's current role is: LEADER
I20260808 14:32:55.847499 27962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03 [sys.catalog]: This master's current role is: LEADER
I20260808 14:32:55.847952 27969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:32:55.850943 27969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:32:55.856500 27969 catalog_manager.cc:1347] Generated new cluster ID: ab324bdaaff745d183233de701b64b9b
I20260808 14:32:55.856568 27969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:32:55.866716 27969 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:32:55.867700 27969 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:32:55.874732 27969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03: Generated new TSK 0
I20260808 14:32:55.875507 27969 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.125.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
Entry for principal kudu/127.7.125.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab.
I20260808 14:32:55.912297 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:39481
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:41827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:32:56.029433 27981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:56.029704 27981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:56.029793 27981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:56.034529 27981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:56.034595 27981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:56.034690 27981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:32:56.040454 27981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:41827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.65.keytab
--principal=kudu/127.7.125.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:39481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:56.041589 27981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:56.042652 27981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:32:56.053148 27981 server_base.cc:1034] running on GCE node
W20260808 14:32:56.053089 27990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:56.053092 27988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:56.053130 27987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:56.053761 27981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:56.054481 27981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:56.055665 27981 hybrid_clock.cc:648] HybridClock initialized: now 1786199576055648 us; error 31 us; skew 500 ppm
Aug 08 14:32:56 dist-test-slave-w206 krb5kdc[27869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199576, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:32:56.058540 27981 init.cc:377] Logged in from keytab as kudu/127.7.125.65@KRBTEST.COM (short username kudu)
I20260808 14:32:56.059581 27981 webserver.cc:460] Webserver started at http://127.7.125.65:40781/ using document root <none> and password file <none>
I20260808 14:32:56.060214 27981 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:56.060310 27981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:56.060556 27981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:56.062633 27981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "7fdb1778b4684a17b98b6352380b1a3a"
format_stamp: "Formatted at 2026-08-08 14:32:56 on dist-test-slave-w206"
server_key: "c083a1a59ef8e43466762bfa2593eb9a"
server_key_iv: "4a23f3d1c8003758e4ddaf6d53303f6e"
server_key_version: "encryptionkey@0"
I20260808 14:32:56.063231 27981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "7fdb1778b4684a17b98b6352380b1a3a"
format_stamp: "Formatted at 2026-08-08 14:32:56 on dist-test-slave-w206"
server_key: "c083a1a59ef8e43466762bfa2593eb9a"
server_key_iv: "4a23f3d1c8003758e4ddaf6d53303f6e"
server_key_version: "encryptionkey@0"
I20260808 14:32:56.067353 27981 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:32:56.069890 27998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:56.071269 27981 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:32:56.071430 27981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "7fdb1778b4684a17b98b6352380b1a3a"
format_stamp: "Formatted at 2026-08-08 14:32:56 on dist-test-slave-w206"
server_key: "c083a1a59ef8e43466762bfa2593eb9a"
server_key_iv: "4a23f3d1c8003758e4ddaf6d53303f6e"
server_key_version: "encryptionkey@0"
I20260808 14:32:56.071554 27981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:56.100184 27981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:56.103420 27981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:56.103632 27981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:56.104256 27981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:56.105118 27981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:32:56.105190 27981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:56.105263 27981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:32:56.105314 27981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:56.114790 27981 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:34795
I20260808 14:32:56.114816 28117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:34795 every 8 connection(s)
I20260808 14:32:56.115839 27981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:32:56.120075 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 27981
I20260808 14:32:56.120280 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:32:56.120600 7669 external_mini_cluster.cc:1428] Setting key eaa98b8fb4d2ce1e4c5c01d00fb9c1b0
Aug 08 14:32:56 dist-test-slave-w206 krb5kdc[27869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199576, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.65@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:32:56.131162 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:56.117735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:40695 (local address 127.7.125.126:39481)
0808 14:32:56.117892 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:56.117895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:56.118708 (+ 813us) server_negotiation.cc:407] Connection header received
0808 14:32:56.120177 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:56.120180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:56.120242 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:56.120351 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:56.121714 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.122388 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.123635 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.123875 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.123988 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:56.126941 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:56.126946 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:56.126948 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:56.127036 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:56.128914 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:56.129485 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:56.129488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:56.129490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:56.129581 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:56.129973 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:56.129977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:56.129979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:56.130184 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:56.130296 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:56.130815 (+ 519us) server_negotiation.cc:299] Negotiation successful
0808 14:32:56.130966 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
WARNING: no policy specified for kudu/127.7.125.66@KRBTEST.COM; defaulting to no policy
I20260808 14:32:56.132050 28120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:56.118074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:39481 (local address 127.7.125.65:40695)
0808 14:32:56.118546 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:56.118591 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:56.119946 (+ 1355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:56.120534 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:56.120541 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:56.121005 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:56.121559 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:56.121574 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.122527 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.122531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:56.123505 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:56.123515 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.124034 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.124039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:56.124212 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:56.124843 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:32:56.124866 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:56.126699 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:56.129056 (+ 2357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:56.129063 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:56.129082 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:56.129369 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:56.129682 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:56.129686 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:56.129688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:56.129870 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:56.130295 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:56.130304 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:56.130574 (+ 270us) client_negotiation.cc:769] Sending connection context
0808 14:32:56.130775 (+ 201us) client_negotiation.cc:240] Negotiation successful
0808 14:32:56.131166 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":129,"threads_started":1}
I20260808 14:32:56.133666 28118 heartbeater.cc:347] Connected to a master server at 127.7.125.126:39481
I20260808 14:32:56.133976 28118 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.66@KRBTEST.COM" created.
I20260808 14:32:56.134660 28118 heartbeater.cc:511] Master 127.7.125.126:39481 requested a full tablet report, sending...
I20260808 14:32:56.136343 27916 ts_manager.cc:194] Registered new tserver with Master: 7fdb1778b4684a17b98b6352380b1a3a (127.7.125.65:34795)
I20260808 14:32:56.137530 27916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.65@KRBTEST.COM'} at 127.7.125.65:40695
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
Entry for principal kudu/127.7.125.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab.
I20260808 14:32:56.153673 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:39481
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:41827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:32:56.271230 28123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:56.271488 28123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:56.271531 28123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:56.275601 28123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:56.275724 28123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:56.275825 28123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:32:56.281273 28123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:41827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.66.keytab
--principal=kudu/127.7.125.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:39481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:56.282437 28123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:56.283543 28123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:56.293548 28128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:56.293537 28129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:56.293537 28131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:56.293900 28123 server_base.cc:1034] running on GCE node
I20260808 14:32:56.294304 28123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:56.295063 28123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:56.296243 28123 hybrid_clock.cc:648] HybridClock initialized: now 1786199576296228 us; error 29 us; skew 500 ppm
Aug 08 14:32:56 dist-test-slave-w206 krb5kdc[27869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199576, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:32:56.299157 28123 init.cc:377] Logged in from keytab as kudu/127.7.125.66@KRBTEST.COM (short username kudu)
I20260808 14:32:56.300060 28123 webserver.cc:460] Webserver started at http://127.7.125.66:39235/ using document root <none> and password file <none>
I20260808 14:32:56.300696 28123 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:56.300793 28123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:56.301035 28123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:56.303145 28123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "9082e810639e4332bb42fb7ab7917b33"
format_stamp: "Formatted at 2026-08-08 14:32:56 on dist-test-slave-w206"
server_key: "d4197c0d4a0a6e7e6aa7e89f82b05cef"
server_key_iv: "9a21aba7eae2c55d722deac7449f91ef"
server_key_version: "encryptionkey@0"
I20260808 14:32:56.303714 28123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "9082e810639e4332bb42fb7ab7917b33"
format_stamp: "Formatted at 2026-08-08 14:32:56 on dist-test-slave-w206"
server_key: "d4197c0d4a0a6e7e6aa7e89f82b05cef"
server_key_iv: "9a21aba7eae2c55d722deac7449f91ef"
server_key_version: "encryptionkey@0"
I20260808 14:32:56.307440 28123 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 14:32:56.309880 28139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:56.311208 28123 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:32:56.311338 28123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "9082e810639e4332bb42fb7ab7917b33"
format_stamp: "Formatted at 2026-08-08 14:32:56 on dist-test-slave-w206"
server_key: "d4197c0d4a0a6e7e6aa7e89f82b05cef"
server_key_iv: "9a21aba7eae2c55d722deac7449f91ef"
server_key_version: "encryptionkey@0"
I20260808 14:32:56.311450 28123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:56.320773 28123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:56.323939 28123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:56.324144 28123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:56.324744 28123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:56.325587 28123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:32:56.325656 28123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:56.325727 28123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:32:56.325776 28123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:56.335023 28123 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:38793
I20260808 14:32:56.335055 28257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:38793 every 8 connection(s)
I20260808 14:32:56.335965 28123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:32:56.341188 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 28123
I20260808 14:32:56.341291 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:32:56.341593 7669 external_mini_cluster.cc:1428] Setting key fe33562760204454408dc2b5a89a76c5
Aug 08 14:32:56 dist-test-slave-w206 krb5kdc[27869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199576, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.66@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:32:56.351696 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:56.337868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:35985 (local address 127.7.125.126:39481)
0808 14:32:56.338047 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:56.338053 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:56.338897 (+ 844us) server_negotiation.cc:407] Connection header received
0808 14:32:56.340464 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:56.340466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:56.340517 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:56.340594 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:56.341918 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.342552 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.343878 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.344104 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.344216 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:56.347589 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:56.347592 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:56.347594 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:56.347618 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:56.349261 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:56.349937 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:56.349939 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:56.349941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:56.350022 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:56.350426 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:56.350428 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:56.350429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:56.350604 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:56.350675 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:56.351418 (+ 743us) server_negotiation.cc:299] Negotiation successful
0808 14:32:56.351557 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
WARNING: no policy specified for kudu/127.7.125.67@KRBTEST.COM; defaulting to no policy
I20260808 14:32:56.352885 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:56.338257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:39481 (local address 127.7.125.66:35985)
0808 14:32:56.338726 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:56.338776 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:56.340208 (+ 1432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:56.340760 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:56.340768 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:56.341145 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:56.341769 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:56.341785 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.342684 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.342687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:56.343749 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:56.343759 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.344270 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.344275 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:56.344446 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:56.345320 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:32:56.345336 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:56.347334 (+ 1998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:56.349389 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:56.349398 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:56.349421 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:56.349827 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:56.350113 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:56.350118 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:56.350122 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:56.350327 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:56.350707 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:56.350717 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:56.351103 (+ 386us) client_negotiation.cc:769] Sending connection context
0808 14:32:56.351340 (+ 237us) client_negotiation.cc:240] Negotiation successful
0808 14:32:56.351713 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":126,"threads_started":1}
I20260808 14:32:56.354135 28258 heartbeater.cc:347] Connected to a master server at 127.7.125.126:39481
I20260808 14:32:56.354359 28258 heartbeater.cc:464] Registering TS with master...
I20260808 14:32:56.354925 28258 heartbeater.cc:511] Master 127.7.125.126:39481 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.67@KRBTEST.COM" created.
I20260808 14:32:56.355983 27916 ts_manager.cc:194] Registered new tserver with Master: 9082e810639e4332bb42fb7ab7917b33 (127.7.125.66:38793)
I20260808 14:32:56.356568 27916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.66@KRBTEST.COM'} at 127.7.125.66:35985
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
Entry for principal kudu/127.7.125.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab.
I20260808 14:32:56.374018 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:39481
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:41827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:32:56.488750 28263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:56.489070 28263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:56.489148 28263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:56.493634 28263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:56.493746 28263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:56.493842 28263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:32:56.499449 28263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:41827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.67.keytab
--principal=kudu/127.7.125.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:39481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:56.500731 28263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:56.501848 28263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:56.511672 28270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:56.511737 28269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:56.511735 28272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:56.511967 28263 server_base.cc:1034] running on GCE node
I20260808 14:32:56.512514 28263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:56.513186 28263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:56.514402 28263 hybrid_clock.cc:648] HybridClock initialized: now 1786199576514369 us; error 44 us; skew 500 ppm
Aug 08 14:32:56 dist-test-slave-w206 krb5kdc[27869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199576, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:32:56.517617 28263 init.cc:377] Logged in from keytab as kudu/127.7.125.67@KRBTEST.COM (short username kudu)
I20260808 14:32:56.518487 28263 webserver.cc:460] Webserver started at http://127.7.125.67:42541/ using document root <none> and password file <none>
I20260808 14:32:56.519112 28263 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:56.519183 28263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:56.519366 28263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:56.521510 28263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "2921356b9c6a4d918a5ee10542845ecc"
format_stamp: "Formatted at 2026-08-08 14:32:56 on dist-test-slave-w206"
server_key: "e5a873bbacb9c090eaf9dbbceb52cf93"
server_key_iv: "3f2f2fe88b06e5917ce92117e235389b"
server_key_version: "encryptionkey@0"
I20260808 14:32:56.522056 28263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "2921356b9c6a4d918a5ee10542845ecc"
format_stamp: "Formatted at 2026-08-08 14:32:56 on dist-test-slave-w206"
server_key: "e5a873bbacb9c090eaf9dbbceb52cf93"
server_key_iv: "3f2f2fe88b06e5917ce92117e235389b"
server_key_version: "encryptionkey@0"
I20260808 14:32:56.525935 28263 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:32:56.528611 28280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:56.529894 28263 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:32:56.530045 28263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "2921356b9c6a4d918a5ee10542845ecc"
format_stamp: "Formatted at 2026-08-08 14:32:56 on dist-test-slave-w206"
server_key: "e5a873bbacb9c090eaf9dbbceb52cf93"
server_key_iv: "3f2f2fe88b06e5917ce92117e235389b"
server_key_version: "encryptionkey@0"
I20260808 14:32:56.530167 28263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:56.549225 28263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:56.552552 28263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:56.552793 28263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:56.553436 28263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:56.554328 28263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:32:56.554401 28263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:56.554472 28263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:32:56.554526 28263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:56.563459 28263 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:38237
I20260808 14:32:56.563527 28399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:38237 every 8 connection(s)
I20260808 14:32:56.564428 28263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:32:56.571777 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 28263
I20260808 14:32:56.571935 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:32:56.572275 7669 external_mini_cluster.cc:1428] Setting key cf8259918693eabac0d3f196c178e5b9
Aug 08 14:32:56 dist-test-slave-w206 krb5kdc[27869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199576, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.67@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:32:56.578809 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:56.566314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:60549 (local address 127.7.125.126:39481)
0808 14:32:56.566470 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:56.566475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:56.567440 (+ 965us) server_negotiation.cc:407] Connection header received
0808 14:32:56.568732 (+ 1292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:56.568736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:56.568788 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:56.568867 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:56.570071 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.570632 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.572285 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.572613 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.572724 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:56.575434 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:56.575438 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:56.575441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:56.575468 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:56.576838 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:56.577362 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:56.577365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:56.577366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:56.577453 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:56.577808 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:56.577811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:56.577812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:56.578013 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:56.578102 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:56.578568 (+ 466us) server_negotiation.cc:299] Negotiation successful
0808 14:32:56.578675 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 14:32:56.579643 28402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:56.566754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:39481 (local address 127.7.125.67:60549)
0808 14:32:56.567285 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:56.567338 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:56.568519 (+ 1181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:56.569033 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:56.569040 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:56.569417 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:56.569929 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:56.569943 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.570743 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.570746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:56.572150 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:56.572162 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.572779 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.572784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:56.572963 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:56.573485 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:32:56.573501 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:56.575222 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:56.576970 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:56.576976 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:56.576989 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:56.577250 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:56.577536 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:32:56.577539 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:32:56.577540 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:32:56.577675 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:32:56.578106 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:56.578110 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:56.578341 (+ 231us) client_negotiation.cc:769] Sending connection context
0808 14:32:56.578498 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 14:32:56.578756 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":112,"threads_started":1}
I20260808 14:32:56.580790 28400 heartbeater.cc:347] Connected to a master server at 127.7.125.126:39481
I20260808 14:32:56.581043 28400 heartbeater.cc:464] Registering TS with master...
I20260808 14:32:56.581530 28400 heartbeater.cc:511] Master 127.7.125.126:39481 requested a full tablet report, sending...
I20260808 14:32:56.582540 27916 ts_manager.cc:194] Registered new tserver with Master: 2921356b9c6a4d918a5ee10542845ecc (127.7.125.67:38237)
I20260808 14:32:56.583166 27916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.125.67@KRBTEST.COM'} at 127.7.125.67:60549
I20260808 14:32:56.586630 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:32:56.596454 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:56.588951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43196 (local address 127.7.125.126:39481)
0808 14:32:56.589110 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:56.589115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:56.589322 (+ 207us) server_negotiation.cc:407] Connection header received
0808 14:32:56.589428 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:56.589430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:56.589472 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:56.589540 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:56.590221 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.590722 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.591626 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.591850 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.591930 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:56.593290 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:56.593294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:56.593296 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:56.593320 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:56.594914 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:56.595354 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:56.595356 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:56.595357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:56.595431 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:56.595657 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:56.595659 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:56.595660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:56.595892 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:56.596001 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:56.596162 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 14:32:56.596260 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 14:32:56.599597 27916 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:32:56.602690 27916 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:32:56.618122 28412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:56.611897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:34795 (local address 127.0.0.1:52336)
0808 14:32:56.612435 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:56.612488 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:56.612617 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:56.613418 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:56.613426 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:56.613453 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:56.613594 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:56.613602 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.614970 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.615007 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:56.616582 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:56.616592 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.617577 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.617581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:56.617686 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:56.617724 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 14:32:56.617833 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 14:32:56.617965 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":5,"thread_start_us":119,"threads_started":1}
I20260808 14:32:56.618350 28416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:56.612712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:38793 (local address 127.0.0.1:33828)
0808 14:32:56.613279 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:56.613293 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:56.613385 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:56.614180 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:56.614183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:56.614204 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:56.614302 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:56.614307 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.615590 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.615594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:56.616720 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:56.616727 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.617888 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.617891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:56.618043 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:56.618077 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 14:32:56.618136 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 14:32:56.618189 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"mutex_wait_us":4,"thread_start_us":92,"threads_started":1}
I20260808 14:32:56.618227 28413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:56.612103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52336 (local address 127.7.125.65:34795)
0808 14:32:56.612506 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:56.612523 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:56.612550 (+ 27us) server_negotiation.cc:407] Connection header received
0808 14:32:56.612829 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:56.612835 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:56.613072 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:56.613220 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:56.613726 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.614822 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.616733 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.617445 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.617577 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:56.617746 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:56.617976 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 14:32:56.618079 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":115,"threads_started":1}
I20260808 14:32:56.618757 28417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:56.612849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41576 (local address 127.7.125.67:38237)
0808 14:32:56.613613 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:56.613625 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:56.613646 (+ 21us) server_negotiation.cc:407] Connection header received
0808 14:32:56.613720 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:56.613724 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:56.613866 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:56.614009 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:56.614236 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.615282 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.616713 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.617571 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.617680 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:56.617812 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:56.618351 (+ 539us) server_negotiation.cc:299] Negotiation successful
0808 14:32:56.618614 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":150,"threads_started":1}
I20260808 14:32:56.618760 28418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:56.613127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33828 (local address 127.7.125.66:38793)
0808 14:32:56.613578 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:56.613589 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:56.613610 (+ 21us) server_negotiation.cc:407] Connection header received
0808 14:32:56.613687 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:56.613692 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:56.613858 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:56.614009 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:56.614430 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.615447 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.616818 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.617585 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.617680 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:56.617812 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:56.618521 (+ 709us) server_negotiation.cc:299] Negotiation successful
0808 14:32:56.618628 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":122,"threads_started":1}
I20260808 14:32:56.621322 28048 tablet_service.cc:1467] Processing CreateTablet for tablet 099034a121c7468c96e2e55ee31b5e43 (DEFAULT_TABLE table=test-table [id=fc1ed4ff23e342b7b8231581aa2ce460]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:32:56.621321 28189 tablet_service.cc:1467] Processing CreateTablet for tablet 099034a121c7468c96e2e55ee31b5e43 (DEFAULT_TABLE table=test-table [id=fc1ed4ff23e342b7b8231581aa2ce460]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:32:56.621932 28415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:56.612263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:38237 (local address 127.0.0.1:41576)
0808 14:32:56.612867 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:56.612880 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:56.612956 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:56.614010 (+ 1054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:56.614013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:56.614032 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:56.614125 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:56.614131 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.615431 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.615434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:56.616584 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:56.616593 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.617826 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.617830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:56.617990 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:56.618017 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 14:32:56.618091 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 14:32:56.618178 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"mutex_wait_us":84,"thread_start_us":118,"threads_started":1}
I20260808 14:32:56.622342 28048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 099034a121c7468c96e2e55ee31b5e43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:56.622350 28189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 099034a121c7468c96e2e55ee31b5e43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:56.624946 28331 tablet_service.cc:1467] Processing CreateTablet for tablet 099034a121c7468c96e2e55ee31b5e43 (DEFAULT_TABLE table=test-table [id=fc1ed4ff23e342b7b8231581aa2ce460]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:32:56.626032 28331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 099034a121c7468c96e2e55ee31b5e43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:56.629122 28419 tablet_bootstrap.cc:492] T 099034a121c7468c96e2e55ee31b5e43 P 9082e810639e4332bb42fb7ab7917b33: Bootstrap starting.
I20260808 14:32:56.631444 28421 tablet_bootstrap.cc:492] T 099034a121c7468c96e2e55ee31b5e43 P 7fdb1778b4684a17b98b6352380b1a3a: Bootstrap starting.
I20260808 14:32:56.632200 28419 tablet_bootstrap.cc:654] T 099034a121c7468c96e2e55ee31b5e43 P 9082e810639e4332bb42fb7ab7917b33: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:56.633105 28419 log.cc:824] T 099034a121c7468c96e2e55ee31b5e43 P 9082e810639e4332bb42fb7ab7917b33: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:56.633438 28422 tablet_bootstrap.cc:492] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc: Bootstrap starting.
I20260808 14:32:56.633774 28421 tablet_bootstrap.cc:654] T 099034a121c7468c96e2e55ee31b5e43 P 7fdb1778b4684a17b98b6352380b1a3a: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:56.634660 28421 log.cc:824] T 099034a121c7468c96e2e55ee31b5e43 P 7fdb1778b4684a17b98b6352380b1a3a: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:56.635826 28422 tablet_bootstrap.cc:654] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:56.636523 28421 tablet_bootstrap.cc:492] T 099034a121c7468c96e2e55ee31b5e43 P 7fdb1778b4684a17b98b6352380b1a3a: No bootstrap required, opened a new log
I20260808 14:32:56.636632 28419 tablet_bootstrap.cc:492] T 099034a121c7468c96e2e55ee31b5e43 P 9082e810639e4332bb42fb7ab7917b33: No bootstrap required, opened a new log
I20260808 14:32:56.636770 28421 ts_tablet_manager.cc:1397] T 099034a121c7468c96e2e55ee31b5e43 P 7fdb1778b4684a17b98b6352380b1a3a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 14:32:56.636835 28419 ts_tablet_manager.cc:1397] T 099034a121c7468c96e2e55ee31b5e43 P 9082e810639e4332bb42fb7ab7917b33: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260808 14:32:56.637071 28422 log.cc:824] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:56.639084 28422 tablet_bootstrap.cc:492] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc: No bootstrap required, opened a new log
I20260808 14:32:56.639276 28422 ts_tablet_manager.cc:1397] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 14:32:56.640559 28421 raft_consensus.cc:348] T 099034a121c7468c96e2e55ee31b5e43 P 7fdb1778b4684a17b98b6352380b1a3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fdb1778b4684a17b98b6352380b1a3a" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34795 } } peers { permanent_uuid: "9082e810639e4332bb42fb7ab7917b33" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38793 } } peers { permanent_uuid: "2921356b9c6a4d918a5ee10542845ecc" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 38237 } }
I20260808 14:32:56.640863 28421 raft_consensus.cc:374] T 099034a121c7468c96e2e55ee31b5e43 P 7fdb1778b4684a17b98b6352380b1a3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:56.640919 28421 raft_consensus.cc:729] T 099034a121c7468c96e2e55ee31b5e43 P 7fdb1778b4684a17b98b6352380b1a3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fdb1778b4684a17b98b6352380b1a3a, State: Initialized, Role: FOLLOWER
I20260808 14:32:56.641414 28421 consensus_queue.cc:260] T 099034a121c7468c96e2e55ee31b5e43 P 7fdb1778b4684a17b98b6352380b1a3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fdb1778b4684a17b98b6352380b1a3a" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34795 } } peers { permanent_uuid: "9082e810639e4332bb42fb7ab7917b33" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38793 } } peers { permanent_uuid: "2921356b9c6a4d918a5ee10542845ecc" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 38237 } }
I20260808 14:32:56.641227 28419 raft_consensus.cc:348] T 099034a121c7468c96e2e55ee31b5e43 P 9082e810639e4332bb42fb7ab7917b33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fdb1778b4684a17b98b6352380b1a3a" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34795 } } peers { permanent_uuid: "9082e810639e4332bb42fb7ab7917b33" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38793 } } peers { permanent_uuid: "2921356b9c6a4d918a5ee10542845ecc" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 38237 } }
I20260808 14:32:56.641637 28419 raft_consensus.cc:374] T 099034a121c7468c96e2e55ee31b5e43 P 9082e810639e4332bb42fb7ab7917b33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:56.641693 28419 raft_consensus.cc:729] T 099034a121c7468c96e2e55ee31b5e43 P 9082e810639e4332bb42fb7ab7917b33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9082e810639e4332bb42fb7ab7917b33, State: Initialized, Role: FOLLOWER
I20260808 14:32:56.642132 28419 consensus_queue.cc:260] T 099034a121c7468c96e2e55ee31b5e43 P 9082e810639e4332bb42fb7ab7917b33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fdb1778b4684a17b98b6352380b1a3a" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34795 } } peers { permanent_uuid: "9082e810639e4332bb42fb7ab7917b33" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38793 } } peers { permanent_uuid: "2921356b9c6a4d918a5ee10542845ecc" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 38237 } }
I20260808 14:32:56.642506 28421 ts_tablet_manager.cc:1428] T 099034a121c7468c96e2e55ee31b5e43 P 7fdb1778b4684a17b98b6352380b1a3a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 14:32:56.642894 28118 heartbeater.cc:503] Master 127.7.125.126:39481 was elected leader, sending a full tablet report...
I20260808 14:32:56.643180 28419 ts_tablet_manager.cc:1428] T 099034a121c7468c96e2e55ee31b5e43 P 9082e810639e4332bb42fb7ab7917b33: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 14:32:56.643214 28422 raft_consensus.cc:348] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fdb1778b4684a17b98b6352380b1a3a" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34795 } } peers { permanent_uuid: "9082e810639e4332bb42fb7ab7917b33" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38793 } } peers { permanent_uuid: "2921356b9c6a4d918a5ee10542845ecc" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 38237 } }
I20260808 14:32:56.643486 28422 raft_consensus.cc:374] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:56.643570 28422 raft_consensus.cc:729] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2921356b9c6a4d918a5ee10542845ecc, State: Initialized, Role: FOLLOWER
I20260808 14:32:56.643962 28422 consensus_queue.cc:260] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fdb1778b4684a17b98b6352380b1a3a" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34795 } } peers { permanent_uuid: "9082e810639e4332bb42fb7ab7917b33" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38793 } } peers { permanent_uuid: "2921356b9c6a4d918a5ee10542845ecc" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 38237 } }
I20260808 14:32:56.644114 28258 heartbeater.cc:503] Master 127.7.125.126:39481 was elected leader, sending a full tablet report...
I20260808 14:32:56.644761 28422 ts_tablet_manager.cc:1428] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 14:32:56.645125 28400 heartbeater.cc:503] Master 127.7.125.126:39481 was elected leader, sending a full tablet report...
I20260808 14:32:56.686481 28431 raft_consensus.cc:482] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:32:56.686805 28431 raft_consensus.cc:504] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fdb1778b4684a17b98b6352380b1a3a" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34795 } } peers { permanent_uuid: "9082e810639e4332bb42fb7ab7917b33" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38793 } } peers { permanent_uuid: "2921356b9c6a4d918a5ee10542845ecc" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 38237 } }
I20260808 14:32:56.688248 28431 leader_election.cc:288] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fdb1778b4684a17b98b6352380b1a3a (127.7.125.65:34795), 9082e810639e4332bb42fb7ab7917b33 (127.7.125.66:38793)
I20260808 14:32:56.692560 28418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:56.689044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:46297 (local address 127.7.125.66:38793)
0808 14:32:56.689177 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:56.689182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:56.689242 (+ 60us) server_negotiation.cc:407] Connection header received
0808 14:32:56.689372 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:56.689375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:56.689432 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:56.689554 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:56.689877 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.690379 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.691580 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.691975 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.692110 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:56.692166 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:56.692373 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 14:32:56.692443 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 14:32:56.692561 28402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:56.688504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:34795 (local address 127.7.125.67:46261)
0808 14:32:56.688618 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:56.688636 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:56.688739 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:56.689285 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:56.689288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:56.689317 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:56.689476 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:56.689488 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.690478 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.690481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:56.691413 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:56.691422 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.692082 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.692086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:56.692197 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:56.692219 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 14:32:56.692292 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 14:32:56.692390 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":2}
I20260808 14:32:56.692579 28413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:56.688681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:46261 (local address 127.7.125.65:34795)
0808 14:32:56.688939 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:56.688944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:56.688960 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:32:56.689014 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:56.689019 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:56.689074 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:56.689155 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:56.689613 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.690358 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.691542 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.691954 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.692075 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:56.692130 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:56.692346 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 14:32:56.692459 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260808 14:32:56.692600 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:56.688755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:38793 (local address 127.7.125.67:46297)
0808 14:32:56.689173 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:56.689186 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:56.689277 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:56.689609 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:56.689612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:56.689629 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:56.689746 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:56.689751 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.690522 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.690525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:56.691423 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:56.691433 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.692117 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.692120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:56.692222 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:56.692239 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 14:32:56.692333 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 14:32:56.692437 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":26,"thread_start_us":147,"threads_started":1}
I20260808 14:32:56.693257 28071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099034a121c7468c96e2e55ee31b5e43" candidate_uuid: "2921356b9c6a4d918a5ee10542845ecc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fdb1778b4684a17b98b6352380b1a3a" is_pre_election: true
I20260808 14:32:56.693257 28209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099034a121c7468c96e2e55ee31b5e43" candidate_uuid: "2921356b9c6a4d918a5ee10542845ecc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9082e810639e4332bb42fb7ab7917b33" is_pre_election: true
I20260808 14:32:56.693584 28209 raft_consensus.cc:2454] T 099034a121c7468c96e2e55ee31b5e43 P 9082e810639e4332bb42fb7ab7917b33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2921356b9c6a4d918a5ee10542845ecc in term 0.
I20260808 14:32:56.693584 28071 raft_consensus.cc:2454] T 099034a121c7468c96e2e55ee31b5e43 P 7fdb1778b4684a17b98b6352380b1a3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2921356b9c6a4d918a5ee10542845ecc in term 0.
I20260808 14:32:56.694078 28283 leader_election.cc:302] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2921356b9c6a4d918a5ee10542845ecc, 7fdb1778b4684a17b98b6352380b1a3a; no voters:
I20260808 14:32:56.694331 28431 raft_consensus.cc:2781] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:32:56.694415 28431 raft_consensus.cc:482] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:32:56.694448 28431 raft_consensus.cc:3037] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:56.695523 28431 raft_consensus.cc:504] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fdb1778b4684a17b98b6352380b1a3a" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34795 } } peers { permanent_uuid: "9082e810639e4332bb42fb7ab7917b33" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38793 } } peers { permanent_uuid: "2921356b9c6a4d918a5ee10542845ecc" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 38237 } }
I20260808 14:32:56.695870 28431 leader_election.cc:288] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc [CANDIDATE]: Term 1 election: Requested vote from peers 7fdb1778b4684a17b98b6352380b1a3a (127.7.125.65:34795), 9082e810639e4332bb42fb7ab7917b33 (127.7.125.66:38793)
I20260808 14:32:56.696390 28071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099034a121c7468c96e2e55ee31b5e43" candidate_uuid: "2921356b9c6a4d918a5ee10542845ecc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fdb1778b4684a17b98b6352380b1a3a"
I20260808 14:32:56.696403 28209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099034a121c7468c96e2e55ee31b5e43" candidate_uuid: "2921356b9c6a4d918a5ee10542845ecc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9082e810639e4332bb42fb7ab7917b33"
I20260808 14:32:56.696533 28209 raft_consensus.cc:3037] T 099034a121c7468c96e2e55ee31b5e43 P 9082e810639e4332bb42fb7ab7917b33 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:56.696504 28071 raft_consensus.cc:3037] T 099034a121c7468c96e2e55ee31b5e43 P 7fdb1778b4684a17b98b6352380b1a3a [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:56.697602 28071 raft_consensus.cc:2454] T 099034a121c7468c96e2e55ee31b5e43 P 7fdb1778b4684a17b98b6352380b1a3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2921356b9c6a4d918a5ee10542845ecc in term 1.
I20260808 14:32:56.697613 28209 raft_consensus.cc:2454] T 099034a121c7468c96e2e55ee31b5e43 P 9082e810639e4332bb42fb7ab7917b33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2921356b9c6a4d918a5ee10542845ecc in term 1.
I20260808 14:32:56.697921 28283 leader_election.cc:302] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2921356b9c6a4d918a5ee10542845ecc, 7fdb1778b4684a17b98b6352380b1a3a; no voters:
I20260808 14:32:56.698101 28431 raft_consensus.cc:2781] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:32:56.698357 28431 raft_consensus.cc:686] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc [term 1 LEADER]: Becoming Leader. State: Replica: 2921356b9c6a4d918a5ee10542845ecc, State: Running, Role: LEADER
I20260808 14:32:56.698691 28431 consensus_queue.cc:237] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fdb1778b4684a17b98b6352380b1a3a" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34795 } } peers { permanent_uuid: "9082e810639e4332bb42fb7ab7917b33" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38793 } } peers { permanent_uuid: "2921356b9c6a4d918a5ee10542845ecc" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 38237 } }
I20260808 14:32:56.701993 27915 catalog_manager.cc:5495] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc reported cstate change: term changed from 0 to 1, leader changed from <none> to 2921356b9c6a4d918a5ee10542845ecc (127.7.125.67). New cstate: current_term: 1 leader_uuid: "2921356b9c6a4d918a5ee10542845ecc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fdb1778b4684a17b98b6352380b1a3a" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9082e810639e4332bb42fb7ab7917b33" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2921356b9c6a4d918a5ee10542845ecc" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 38237 } health_report { overall_health: HEALTHY } } }
I20260808 14:32:56.716939 28417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:56.713280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41578 (local address 127.7.125.67:38237)
0808 14:32:56.713401 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:56.713405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:56.713419 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:32:56.713567 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:56.713570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:56.713624 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:56.713710 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:32:56.713954 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.714455 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.715516 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.715723 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:56.715802 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:56.716166 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:32:56.716570 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:32:56.716667 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:56.716750 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 14:32:56.716800 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 14:32:56.722930 28209 raft_consensus.cc:1261] T 099034a121c7468c96e2e55ee31b5e43 P 9082e810639e4332bb42fb7ab7917b33 [term 1 FOLLOWER]: Refusing update from remote peer 2921356b9c6a4d918a5ee10542845ecc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:32:56.723038 28071 raft_consensus.cc:1261] T 099034a121c7468c96e2e55ee31b5e43 P 7fdb1778b4684a17b98b6352380b1a3a [term 1 FOLLOWER]: Refusing update from remote peer 2921356b9c6a4d918a5ee10542845ecc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:32:56.723757 28431 consensus_queue.cc:1035] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fdb1778b4684a17b98b6352380b1a3a" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 34795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:32:56.723942 28433 consensus_queue.cc:1035] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9082e810639e4332bb42fb7ab7917b33" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 38793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:32:56.732336 28437 mvcc.cc:205] Tried to move back new op lower bound from 7316273466250035200 to 7316273466160922624. Current Snapshot: MvccSnapshot[applied={T|T < 7316273466250035200}]
I20260808 14:32:56.741361 27916 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:32:56.741552 27916 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:32:56.744143 27916 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 224126a463964115a0832507fce64f03: Sending DeleteTablet for 3 replicas of tablet 099034a121c7468c96e2e55ee31b5e43
I20260808 14:32:56.744940 28331 tablet_service.cc:1514] Processing DeleteTablet for tablet 099034a121c7468c96e2e55ee31b5e43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:32:56 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:41576
I20260808 14:32:56.745030 28048 tablet_service.cc:1514] Processing DeleteTablet for tablet 099034a121c7468c96e2e55ee31b5e43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:32:56 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:52336
I20260808 14:32:56.745455 28447 tablet_replica.cc:330] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc: stopping tablet replica
I20260808 14:32:56.745452 28189 tablet_service.cc:1514] Processing DeleteTablet for tablet 099034a121c7468c96e2e55ee31b5e43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:32:56 UTC) from {username='kudu', principal='kudu/127.7.125.126@KRBTEST.COM'} at 127.0.0.1:33828
I20260808 14:32:56.745770 28447 raft_consensus.cc:2229] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:32:56.745870 28449 tablet_replica.cc:330] T 099034a121c7468c96e2e55ee31b5e43 P 9082e810639e4332bb42fb7ab7917b33: stopping tablet replica
I20260808 14:32:56.746145 28449 raft_consensus.cc:2229] T 099034a121c7468c96e2e55ee31b5e43 P 9082e810639e4332bb42fb7ab7917b33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:32:56.746197 28447 raft_consensus.cc:2258] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:32:56.746513 28449 raft_consensus.cc:2258] T 099034a121c7468c96e2e55ee31b5e43 P 9082e810639e4332bb42fb7ab7917b33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:32:56.746955 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 27981
I20260808 14:32:56.749558 28449 ts_tablet_manager.cc:1905] T 099034a121c7468c96e2e55ee31b5e43 P 9082e810639e4332bb42fb7ab7917b33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:32:56.752516 28447 ts_tablet_manager.cc:1905] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:32:56.753260 28449 ts_tablet_manager.cc:1918] T 099034a121c7468c96e2e55ee31b5e43 P 9082e810639e4332bb42fb7ab7917b33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:32:56.753372 28449 log.cc:1196] T 099034a121c7468c96e2e55ee31b5e43 P 9082e810639e4332bb42fb7ab7917b33: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/099034a121c7468c96e2e55ee31b5e43
I20260808 14:32:56.753842 28449 ts_tablet_manager.cc:1939] T 099034a121c7468c96e2e55ee31b5e43 P 9082e810639e4332bb42fb7ab7917b33: Deleting consensus metadata
I20260808 14:32:56.755295 27900 catalog_manager.cc:4841] TS 9082e810639e4332bb42fb7ab7917b33 (127.7.125.66:38793): tablet 099034a121c7468c96e2e55ee31b5e43 (table test-table [id=fc1ed4ff23e342b7b8231581aa2ce460]) successfully deleted
I20260808 14:32:56.755731 28447 ts_tablet_manager.cc:1918] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:32:56.755828 28447 log.cc:1196] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/099034a121c7468c96e2e55ee31b5e43
I20260808 14:32:56.756140 28447 ts_tablet_manager.cc:1939] T 099034a121c7468c96e2e55ee31b5e43 P 2921356b9c6a4d918a5ee10542845ecc: Deleting consensus metadata
W20260808 14:32:56.756729 27900 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.125.65:34795 (error 108)
I20260808 14:32:56.756928 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 28123
I20260808 14:32:56.766142 27898 catalog_manager.cc:4841] TS 2921356b9c6a4d918a5ee10542845ecc (127.7.125.67:38237): tablet 099034a121c7468c96e2e55ee31b5e43 (table test-table [id=fc1ed4ff23e342b7b8231581aa2ce460]) successfully deleted
I20260808 14:32:56.766279 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 28263
I20260808 14:32:56.769116 28415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:56.768701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:34795 (local address 127.0.0.1:52352)
0808 14:32:56.768891 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:56.768996 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.65:34795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260808 14:32:56.769354 27900 catalog_manager.cc:4584] TS 7fdb1778b4684a17b98b6352380b1a3a (127.7.125.65:34795): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 099034a121c7468c96e2e55ee31b5e43: Network error: Client connection negotiation failed: client connection to 127.7.125.65:34795: connect: Connection refused (error 111)
I20260808 14:32:56.773454 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 27882
2026-08-08T14:32:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2369 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T14:32:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:32:56Z Disabled control of system clock
I20260808 14:32:56.807300 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:43111
--webserver_interface=127.7.125.126
--webserver_port=0
--builtin_ntp_servers=127.7.125.84:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:43111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 14:32:56.924950 28454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:56.925225 28454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:56.925262 28454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:56.929513 28454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:32:56.929572 28454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:56.929589 28454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:56.929607 28454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:32:56.929661 28454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:32:56.934625 28454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:43111
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:43111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:56.935778 28454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:56.937151 28454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:56.945600 28459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:56.945619 28460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:56.945607 28462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:56.946007 28454 server_base.cc:1034] running on GCE node
I20260808 14:32:56.946592 28454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:56.947728 28454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:56.948916 28454 hybrid_clock.cc:648] HybridClock initialized: now 1786199576948904 us; error 26 us; skew 500 ppm
I20260808 14:32:56.950898 28454 webserver.cc:460] Webserver started at http://127.7.125.126:34977/ using document root <none> and password file <none>
I20260808 14:32:56.951493 28454 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:56.951584 28454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:56.951817 28454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:56.953862 28454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "5479992adddd4f659880ee20121d1b6e"
format_stamp: "Formatted at 2026-08-08 14:32:56 on dist-test-slave-w206"
server_key: "e36fb779763de5c1cbb9d996c33a6004"
server_key_iv: "f64ca149b813821b6bbcb9cb63f4565a"
server_key_version: "encryptionkey@0"
I20260808 14:32:56.954419 28454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "5479992adddd4f659880ee20121d1b6e"
format_stamp: "Formatted at 2026-08-08 14:32:56 on dist-test-slave-w206"
server_key: "e36fb779763de5c1cbb9d996c33a6004"
server_key_iv: "f64ca149b813821b6bbcb9cb63f4565a"
server_key_version: "encryptionkey@0"
I20260808 14:32:56.958129 28454 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 14:32:56.960631 28468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:56.961869 28454 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:32:56.962028 28454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "5479992adddd4f659880ee20121d1b6e"
format_stamp: "Formatted at 2026-08-08 14:32:56 on dist-test-slave-w206"
server_key: "e36fb779763de5c1cbb9d996c33a6004"
server_key_iv: "f64ca149b813821b6bbcb9cb63f4565a"
server_key_version: "encryptionkey@0"
I20260808 14:32:56.962145 28454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:56.979702 28454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:56.980433 28454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:56.980633 28454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:56.987463 28454 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:43111
I20260808 14:32:56.987499 28524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:43111 every 8 connection(s)
I20260808 14:32:56.988397 28454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:32:56.991500 28525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:56.994112 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 28454
I20260808 14:32:56.994258 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:32:56.994571 7669 external_mini_cluster.cc:1428] Setting key c9459d535c17cfebe193f3bce9104a2e
I20260808 14:32:56.997723 28525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e: Bootstrap starting.
I20260808 14:32:57.000013 28525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:57.000743 28525 log.cc:824] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:57.002591 28525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e: No bootstrap required, opened a new log
I20260808 14:32:57.004350 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:56.996112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37758 (local address 127.7.125.126:43111)
0808 14:32:56.996825 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:56.996841 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:56.996878 (+ 37us) server_negotiation.cc:407] Connection header received
0808 14:32:56.997489 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:56.997505 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:56.997954 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:56.998291 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:56.998750 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:56.999830 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.000823 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.001344 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.001523 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:57.002420 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:57.002426 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:57.002439 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:32:57.002883 (+ 444us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:57.002940 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:32:57.002961 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:57.003115 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:57.003287 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 14:32:57.003547 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":148,"threads_started":1}
I20260808 14:32:57.006309 28525 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5479992adddd4f659880ee20121d1b6e" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43111 } }
I20260808 14:32:57.006613 28525 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:57.006702 28525 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5479992adddd4f659880ee20121d1b6e, State: Initialized, Role: FOLLOWER
I20260808 14:32:57.007185 28525 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5479992adddd4f659880ee20121d1b6e" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43111 } }
I20260808 14:32:57.007337 28525 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:32:57.007422 28525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:32:57.007519 28525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:57.008453 28525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5479992adddd4f659880ee20121d1b6e" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43111 } }
I20260808 14:32:57.008778 28525 leader_election.cc:302] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5479992adddd4f659880ee20121d1b6e; no voters:
I20260808 14:32:57.009093 28525 leader_election.cc:288] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:32:57.009217 28530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:32:57.009500 28530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e [term 1 LEADER]: Becoming Leader. State: Replica: 5479992adddd4f659880ee20121d1b6e, State: Running, Role: LEADER
I20260808 14:32:57.009812 28530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5479992adddd4f659880ee20121d1b6e" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43111 } }
I20260808 14:32:57.010298 28525 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:32:57.011524 28531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5479992adddd4f659880ee20121d1b6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5479992adddd4f659880ee20121d1b6e" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43111 } } }
I20260808 14:32:57.011667 28531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e [sys.catalog]: This master's current role is: LEADER
I20260808 14:32:57.012046 28539 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:32:57.011981 28532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5479992adddd4f659880ee20121d1b6e. Latest consensus state: current_term: 1 leader_uuid: "5479992adddd4f659880ee20121d1b6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5479992adddd4f659880ee20121d1b6e" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 43111 } } }
I20260808 14:32:57.012583 28532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e [sys.catalog]: This master's current role is: LEADER
I20260808 14:32:57.015319 28539 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:32:57.020929 28539 catalog_manager.cc:1347] Generated new cluster ID: 77c75e5de712456baf80131f90fe19a9
I20260808 14:32:57.020998 28539 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:32:57.031172 28539 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:32:57.032145 28539 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:32:57.040386 28539 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e: Generated new TSK 0
I20260808 14:32:57.041141 28539 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:32:57.050467 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.65:0
--local_ip_for_outbound_sockets=127.7.125.65
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:43111
--builtin_ntp_servers=127.7.125.84:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 14:32:57.173362 28549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:57.173641 28549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:57.173717 28549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:57.177902 28549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:57.177963 28549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:57.178076 28549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.65
I20260808 14:32:57.183001 28549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.125.65
--webserver_port=0
--tserver_master_addrs=127.7.125.126:43111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.65
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:57.184175 28549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:57.185623 28549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:57.195796 28554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:57.195736 28555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:57.195990 28549 server_base.cc:1034] running on GCE node
W20260808 14:32:57.195727 28557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:57.196736 28549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:57.197432 28549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:57.198625 28549 hybrid_clock.cc:648] HybridClock initialized: now 1786199577198600 us; error 35 us; skew 500 ppm
I20260808 14:32:57.200801 28549 webserver.cc:460] Webserver started at http://127.7.125.65:39527/ using document root <none> and password file <none>
I20260808 14:32:57.201435 28549 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:57.201534 28549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:57.201843 28549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:57.204155 28549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-0/data/instance:
uuid: "3bd12b2bc474498592078d087577541e"
format_stamp: "Formatted at 2026-08-08 14:32:57 on dist-test-slave-w206"
server_key: "09e77c66f745c57bb1d4f278fca8604a"
server_key_iv: "5647a3c04f23d6ef839fc7fa57c976bb"
server_key_version: "encryptionkey@0"
I20260808 14:32:57.204777 28549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance:
uuid: "3bd12b2bc474498592078d087577541e"
format_stamp: "Formatted at 2026-08-08 14:32:57 on dist-test-slave-w206"
server_key: "09e77c66f745c57bb1d4f278fca8604a"
server_key_iv: "5647a3c04f23d6ef839fc7fa57c976bb"
server_key_version: "encryptionkey@0"
I20260808 14:32:57.208819 28549 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:32:57.211598 28563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:57.212949 28549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:32:57.213094 28549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-0/wal
uuid: "3bd12b2bc474498592078d087577541e"
format_stamp: "Formatted at 2026-08-08 14:32:57 on dist-test-slave-w206"
server_key: "09e77c66f745c57bb1d4f278fca8604a"
server_key_iv: "5647a3c04f23d6ef839fc7fa57c976bb"
server_key_version: "encryptionkey@0"
I20260808 14:32:57.213214 28549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:57.233217 28549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:57.234001 28549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:57.234220 28549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:57.234849 28549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:57.235847 28549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:32:57.235920 28549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:57.236001 28549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:32:57.236043 28549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:57.245159 28549 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.65:43841
I20260808 14:32:57.245201 28683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.65:43841 every 8 connection(s)
I20260808 14:32:57.246120 28549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-0/data/info.pb
I20260808 14:32:57.248723 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 28549
I20260808 14:32:57.248869 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-0/wal/instance
I20260808 14:32:57.249173 7669 external_mini_cluster.cc:1428] Setting key 23cd564cdd6fef519bfed852d6824a60
I20260808 14:32:57.251499 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.66:0
--local_ip_for_outbound_sockets=127.7.125.66
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:43111
--builtin_ntp_servers=127.7.125.84:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 14:32:57.256595 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:57.248048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:47243 (local address 127.7.125.126:43111)
0808 14:32:57.248205 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:57.248209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:57.249403 (+ 1194us) server_negotiation.cc:407] Connection header received
0808 14:32:57.251140 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:57.251143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:57.251186 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:57.251246 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:57.252489 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.253010 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.254303 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.254555 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.254674 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:57.255967 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:57.255974 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:57.255979 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:32:57.255999 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:57.256028 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:32:57.256042 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:57.256128 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:57.256296 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 14:32:57.256379 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:32:57.257493 28686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:57.248397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43111 (local address 127.7.125.65:47243)
0808 14:32:57.249208 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:57.249280 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:57.250880 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:57.251481 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:57.251490 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:57.251606 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:57.252333 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:57.252357 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.253147 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.253153 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:57.254150 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:57.254160 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.254719 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.254723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:57.254900 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:57.255607 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:32:57.255608 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:57.255650 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:32:57.255739 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:57.256146 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:57.256154 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:57.256159 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 14:32:57.256301 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 14:32:57.256548 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":112,"threads_started":1}
I20260808 14:32:57.258852 28684 heartbeater.cc:347] Connected to a master server at 127.7.125.126:43111
I20260808 14:32:57.259162 28684 heartbeater.cc:464] Registering TS with master...
I20260808 14:32:57.259903 28684 heartbeater.cc:511] Master 127.7.125.126:43111 requested a full tablet report, sending...
I20260808 14:32:57.261862 28487 ts_manager.cc:194] Registered new tserver with Master: 3bd12b2bc474498592078d087577541e (127.7.125.65:43841)
I20260808 14:32:57.263209 28487 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.125.65:47243
W20260808 14:32:57.366384 28687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:57.366715 28687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:57.366792 28687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:57.370833 28687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:57.370929 28687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:57.371038 28687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.66
I20260808 14:32:57.375810 28687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.125.66
--webserver_port=0
--tserver_master_addrs=127.7.125.126:43111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.66
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:57.376927 28687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:57.378378 28687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:32:57.388408 28687 server_base.cc:1034] running on GCE node
W20260808 14:32:57.388389 28693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:57.388420 28695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:57.388379 28692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:57.389042 28687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:57.389647 28687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:57.390844 28687 hybrid_clock.cc:648] HybridClock initialized: now 1786199577390814 us; error 47 us; skew 500 ppm
I20260808 14:32:57.392940 28687 webserver.cc:460] Webserver started at http://127.7.125.66:36755/ using document root <none> and password file <none>
I20260808 14:32:57.393555 28687 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:57.393651 28687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:57.393896 28687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:57.396068 28687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-1/data/instance:
uuid: "74ecf34d74204b668675caf6a98b4b83"
format_stamp: "Formatted at 2026-08-08 14:32:57 on dist-test-slave-w206"
server_key: "c033c87d7b4ca676c8c80ebfe7bb00ba"
server_key_iv: "be92c8162ceb651139bd2af96b9aa0c2"
server_key_version: "encryptionkey@0"
I20260808 14:32:57.396644 28687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance:
uuid: "74ecf34d74204b668675caf6a98b4b83"
format_stamp: "Formatted at 2026-08-08 14:32:57 on dist-test-slave-w206"
server_key: "c033c87d7b4ca676c8c80ebfe7bb00ba"
server_key_iv: "be92c8162ceb651139bd2af96b9aa0c2"
server_key_version: "encryptionkey@0"
I20260808 14:32:57.400539 28687 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260808 14:32:57.403203 28702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:57.404495 28687 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 14:32:57.404639 28687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-1/wal
uuid: "74ecf34d74204b668675caf6a98b4b83"
format_stamp: "Formatted at 2026-08-08 14:32:57 on dist-test-slave-w206"
server_key: "c033c87d7b4ca676c8c80ebfe7bb00ba"
server_key_iv: "be92c8162ceb651139bd2af96b9aa0c2"
server_key_version: "encryptionkey@0"
I20260808 14:32:57.404762 28687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:57.423944 28687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:57.424680 28687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:57.424901 28687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:57.425513 28687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:57.426548 28687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:32:57.426621 28687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:57.426693 28687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:32:57.426736 28687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:57.435891 28687 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.66:36925
I20260808 14:32:57.435900 28820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.66:36925 every 8 connection(s)
I20260808 14:32:57.436837 28687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-1/data/info.pb
I20260808 14:32:57.439150 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 28687
I20260808 14:32:57.439281 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-1/wal/instance
I20260808 14:32:57.439657 7669 external_mini_cluster.cc:1428] Setting key ea19e25751668c5ce2e22495cd912a90
I20260808 14:32:57.441886 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.125.67:0
--local_ip_for_outbound_sockets=127.7.125.67
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:43111
--builtin_ntp_servers=127.7.125.84:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 14:32:57.447165 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:57.438744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.66:39215 (local address 127.7.125.126:43111)
0808 14:32:57.438917 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:57.438923 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:57.440221 (+ 1298us) server_negotiation.cc:407] Connection header received
0808 14:32:57.441869 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:57.441872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:57.441913 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:57.442014 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:57.443250 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.443766 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.444914 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.445301 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.445420 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:57.446528 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:57.446533 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:57.446536 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:32:57.446550 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:57.446576 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:32:57.446588 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:57.446675 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:57.446845 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 14:32:57.446919 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 14:32:57.447976 28824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:57.439197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43111 (local address 127.7.125.66:39215)
0808 14:32:57.440052 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:57.440106 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:57.441642 (+ 1536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:57.442190 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:57.442201 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:57.442308 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:57.443066 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:57.443088 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.443905 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.443909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:57.444782 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:57.444792 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.445456 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.445459 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:57.445601 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:57.446241 (+ 640us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:32:57.446243 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:57.446282 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:32:57.446344 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:57.446688 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:57.446697 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:57.446701 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 14:32:57.446803 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 14:32:57.447093 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":118,"threads_started":1}
I20260808 14:32:57.449194 28822 heartbeater.cc:347] Connected to a master server at 127.7.125.126:43111
I20260808 14:32:57.449467 28822 heartbeater.cc:464] Registering TS with master...
I20260808 14:32:57.450130 28822 heartbeater.cc:511] Master 127.7.125.126:43111 requested a full tablet report, sending...
I20260808 14:32:57.451300 28487 ts_manager.cc:194] Registered new tserver with Master: 74ecf34d74204b668675caf6a98b4b83 (127.7.125.66:36925)
I20260808 14:32:57.451885 28487 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.125.66:39215
W20260808 14:32:57.560151 28826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:57.560544 28826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:57.560628 28826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:57.564777 28826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:57.564867 28826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:57.564955 28826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.125.67
I20260808 14:32:57.569831 28826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.125.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.125.67
--webserver_port=0
--tserver_master_addrs=127.7.125.126:43111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.125.67
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:57.571072 28826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:57.572556 28826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:57.582562 28831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:57.582571 28832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:57.582654 28826 server_base.cc:1034] running on GCE node
W20260808 14:32:57.582571 28834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:57.583269 28826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:57.583950 28826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:57.585165 28826 hybrid_clock.cc:648] HybridClock initialized: now 1786199577585143 us; error 40 us; skew 500 ppm
I20260808 14:32:57.587208 28826 webserver.cc:460] Webserver started at http://127.7.125.67:39107/ using document root <none> and password file <none>
I20260808 14:32:57.587823 28826 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:57.587903 28826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:57.588166 28826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:57.590277 28826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-2/data/instance:
uuid: "246fa234db95458fa6612eca85da0521"
format_stamp: "Formatted at 2026-08-08 14:32:57 on dist-test-slave-w206"
server_key: "9d7b47fca895cd90d96a3564e8779d28"
server_key_iv: "e6804ee3693459523022e15a22ce4b8c"
server_key_version: "encryptionkey@0"
I20260808 14:32:57.590871 28826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance:
uuid: "246fa234db95458fa6612eca85da0521"
format_stamp: "Formatted at 2026-08-08 14:32:57 on dist-test-slave-w206"
server_key: "9d7b47fca895cd90d96a3564e8779d28"
server_key_iv: "e6804ee3693459523022e15a22ce4b8c"
server_key_version: "encryptionkey@0"
I20260808 14:32:57.594617 28826 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:32:57.597173 28841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:57.598466 28826 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:32:57.598610 28826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-2/wal
uuid: "246fa234db95458fa6612eca85da0521"
format_stamp: "Formatted at 2026-08-08 14:32:57 on dist-test-slave-w206"
server_key: "9d7b47fca895cd90d96a3564e8779d28"
server_key_iv: "e6804ee3693459523022e15a22ce4b8c"
server_key_version: "encryptionkey@0"
I20260808 14:32:57.598726 28826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:57.612180 28826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:57.612946 28826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:57.613159 28826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:57.613761 28826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:32:57.614725 28826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:32:57.614806 28826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:57.614880 28826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:32:57.614915 28826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:57.624222 28826 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.67:39099
I20260808 14:32:57.624254 28963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.67:39099 every 8 connection(s)
I20260808 14:32:57.625195 28826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-2/data/info.pb
I20260808 14:32:57.629670 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 28826
I20260808 14:32:57.629806 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-2/wal/instance
I20260808 14:32:57.630146 7669 external_mini_cluster.cc:1428] Setting key b7516dd682bfe7baf3401f4ec25db702
I20260808 14:32:57.635303 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:57.627127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.67:34231 (local address 127.7.125.126:43111)
0808 14:32:57.627299 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:57.627304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:57.628447 (+ 1143us) server_negotiation.cc:407] Connection header received
0808 14:32:57.630054 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:57.630057 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:57.630108 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:57.630188 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:57.631445 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.632021 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.633306 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.633536 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.633646 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:57.634749 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:57.634753 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:57.634756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:32:57.634775 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:57.634799 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:32:57.634807 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:57.634885 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:57.635080 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 14:32:57.635144 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 14:32:57.636162 28966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:57.627605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:43111 (local address 127.7.125.67:34231)
0808 14:32:57.628282 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:57.628324 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:57.629819 (+ 1495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:57.630408 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:57.630416 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:57.630534 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:32:57.631262 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:57.631284 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.632163 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.632167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:57.633161 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:57.633171 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.633699 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.633704 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:57.633894 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:57.634484 (+ 590us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:32:57.634486 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:32:57.634526 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:32:57.634591 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:32:57.634900 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:32:57.634912 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:32:57.634918 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 14:32:57.635080 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 14:32:57.635369 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":122,"threads_started":1}
I20260808 14:32:57.637244 28964 heartbeater.cc:347] Connected to a master server at 127.7.125.126:43111
I20260808 14:32:57.637501 28964 heartbeater.cc:464] Registering TS with master...
I20260808 14:32:57.638046 28964 heartbeater.cc:511] Master 127.7.125.126:43111 requested a full tablet report, sending...
I20260808 14:32:57.639119 28487 ts_manager.cc:194] Registered new tserver with Master: 246fa234db95458fa6612eca85da0521 (127.7.125.67:39099)
I20260808 14:32:57.639693 28487 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.125.67:34231
I20260808 14:32:57.644124 7669 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:32:57.650022 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:57.646010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37766 (local address 127.7.125.126:43111)
0808 14:32:57.646168 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:57.646173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:57.646358 (+ 185us) server_negotiation.cc:407] Connection header received
0808 14:32:57.646453 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:57.646455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:57.646497 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:57.646562 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:57.647020 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.647537 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.648422 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.648630 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.648706 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:57.649556 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:57.649559 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:57.649561 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:32:57.649573 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:57.649591 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:32:57.649599 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:57.649656 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:57.649766 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 14:32:57.649820 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 14:32:57.653009 28487 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:32:57.656112 28487 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:32:57.670562 28976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:57.664666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:39099 (local address 127.0.0.1:41194)
0808 14:32:57.665351 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:57.665365 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:57.665472 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:57.666297 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:57.666310 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:57.666337 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:57.666487 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:57.666496 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.667739 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.667745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:57.668972 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:57.668982 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.669994 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.669998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:57.670112 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:57.670150 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 14:32:57.670269 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 14:32:57.670395 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"spinlock_wait_cycles":7168,"thread_start_us":124,"threads_started":1}
I20260808 14:32:57.670567 28978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:57.665106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41194 (local address 127.7.125.67:39099)
0808 14:32:57.665575 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:57.665588 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:57.665616 (+ 28us) server_negotiation.cc:407] Connection header received
0808 14:32:57.665701 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:57.665707 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:57.665973 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:57.666137 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:57.666633 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.667625 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.669116 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.669865 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.669993 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:57.670196 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:57.670306 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 14:32:57.670401 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":160,"threads_started":1}
I20260808 14:32:57.671280 28975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:57.664612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:43841 (local address 127.0.0.1:50928)
0808 14:32:57.665262 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:57.665329 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:57.665473 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:57.666620 (+ 1147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:57.666624 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:57.666645 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:57.666750 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:57.666757 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.668306 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.668310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:57.669438 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:57.669448 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.670708 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.670712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:57.670904 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:57.670959 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 14:32:57.671079 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 14:32:57.671131 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":94,"spinlock_wait_cycles":2304,"thread_start_us":102,"threads_started":1}
I20260808 14:32:57.671422 28977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:57.664748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50928 (local address 127.7.125.65:43841)
0808 14:32:57.665792 (+ 1044us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:57.665804 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:57.665826 (+ 22us) server_negotiation.cc:407] Connection header received
0808 14:32:57.665896 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:57.665900 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:57.666055 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:57.666183 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:57.666925 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.668157 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.669623 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.670359 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.670454 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:57.670632 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:57.671132 (+ 500us) server_negotiation.cc:299] Negotiation successful
0808 14:32:57.671257 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":943,"thread_start_us":128,"threads_started":1}
I20260808 14:32:57.671509 28980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:57.665829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41340 (local address 127.7.125.66:36925)
0808 14:32:57.666501 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:57.666513 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:57.666536 (+ 23us) server_negotiation.cc:407] Connection header received
0808 14:32:57.666614 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:57.666619 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:57.666772 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:57.666947 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:57.667158 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.668246 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.669531 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.670344 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.670455 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:57.670632 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:57.671208 (+ 576us) server_negotiation.cc:299] Negotiation successful
0808 14:32:57.671339 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":110,"threads_started":1}
I20260808 14:32:57.671648 28979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:57.665421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:36925 (local address 127.0.0.1:41340)
0808 14:32:57.666350 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:57.666365 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:57.666452 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:57.666928 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:57.666931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:57.666949 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:57.667062 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:57.667068 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.668337 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.668342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:57.669407 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:57.669416 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.670708 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.670712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:57.670899 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:57.670953 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 14:32:57.671048 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 14:32:57.671125 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":831,"mutex_wait_us":91,"thread_start_us":87,"threads_started":1}
I20260808 14:32:57.674175 28891 tablet_service.cc:1467] Processing CreateTablet for tablet 1d72328d4b414860b78db7c65aa14436 (DEFAULT_TABLE table=test-table [id=53aa7f8ef1aa459a98aacd1d185be98d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:32:57.674763 28615 tablet_service.cc:1467] Processing CreateTablet for tablet 1d72328d4b414860b78db7c65aa14436 (DEFAULT_TABLE table=test-table [id=53aa7f8ef1aa459a98aacd1d185be98d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:32:57.675328 28891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d72328d4b414860b78db7c65aa14436. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:57.675856 28615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d72328d4b414860b78db7c65aa14436. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:57.675446 28752 tablet_service.cc:1467] Processing CreateTablet for tablet 1d72328d4b414860b78db7c65aa14436 (DEFAULT_TABLE table=test-table [id=53aa7f8ef1aa459a98aacd1d185be98d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:32:57.676411 28752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d72328d4b414860b78db7c65aa14436. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:57.682853 28981 tablet_bootstrap.cc:492] T 1d72328d4b414860b78db7c65aa14436 P 246fa234db95458fa6612eca85da0521: Bootstrap starting.
I20260808 14:32:57.682853 28982 tablet_bootstrap.cc:492] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e: Bootstrap starting.
I20260808 14:32:57.683836 28983 tablet_bootstrap.cc:492] T 1d72328d4b414860b78db7c65aa14436 P 74ecf34d74204b668675caf6a98b4b83: Bootstrap starting.
I20260808 14:32:57.685321 28981 tablet_bootstrap.cc:654] T 1d72328d4b414860b78db7c65aa14436 P 246fa234db95458fa6612eca85da0521: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:57.685321 28982 tablet_bootstrap.cc:654] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:57.685621 28983 tablet_bootstrap.cc:654] T 1d72328d4b414860b78db7c65aa14436 P 74ecf34d74204b668675caf6a98b4b83: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:57.686266 28981 log.cc:824] T 1d72328d4b414860b78db7c65aa14436 P 246fa234db95458fa6612eca85da0521: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:57.686266 28982 log.cc:824] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:57.686308 28983 log.cc:824] T 1d72328d4b414860b78db7c65aa14436 P 74ecf34d74204b668675caf6a98b4b83: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:57.688562 28982 tablet_bootstrap.cc:492] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e: No bootstrap required, opened a new log
I20260808 14:32:57.688607 28983 tablet_bootstrap.cc:492] T 1d72328d4b414860b78db7c65aa14436 P 74ecf34d74204b668675caf6a98b4b83: No bootstrap required, opened a new log
I20260808 14:32:57.688607 28981 tablet_bootstrap.cc:492] T 1d72328d4b414860b78db7c65aa14436 P 246fa234db95458fa6612eca85da0521: No bootstrap required, opened a new log
I20260808 14:32:57.688802 28982 ts_tablet_manager.cc:1397] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:32:57.688798 28983 ts_tablet_manager.cc:1397] T 1d72328d4b414860b78db7c65aa14436 P 74ecf34d74204b668675caf6a98b4b83: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 14:32:57.688808 28981 ts_tablet_manager.cc:1397] T 1d72328d4b414860b78db7c65aa14436 P 246fa234db95458fa6612eca85da0521: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:32:57.692272 28982 raft_consensus.cc:348] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd12b2bc474498592078d087577541e" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43841 } } peers { permanent_uuid: "246fa234db95458fa6612eca85da0521" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39099 } } peers { permanent_uuid: "74ecf34d74204b668675caf6a98b4b83" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 36925 } }
I20260808 14:32:57.692576 28982 raft_consensus.cc:374] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:57.692662 28982 raft_consensus.cc:729] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd12b2bc474498592078d087577541e, State: Initialized, Role: FOLLOWER
I20260808 14:32:57.693158 28982 consensus_queue.cc:260] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd12b2bc474498592078d087577541e" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43841 } } peers { permanent_uuid: "246fa234db95458fa6612eca85da0521" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39099 } } peers { permanent_uuid: "74ecf34d74204b668675caf6a98b4b83" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 36925 } }
I20260808 14:32:57.693238 28983 raft_consensus.cc:348] T 1d72328d4b414860b78db7c65aa14436 P 74ecf34d74204b668675caf6a98b4b83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd12b2bc474498592078d087577541e" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43841 } } peers { permanent_uuid: "246fa234db95458fa6612eca85da0521" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39099 } } peers { permanent_uuid: "74ecf34d74204b668675caf6a98b4b83" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 36925 } }
I20260808 14:32:57.693238 28981 raft_consensus.cc:348] T 1d72328d4b414860b78db7c65aa14436 P 246fa234db95458fa6612eca85da0521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd12b2bc474498592078d087577541e" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43841 } } peers { permanent_uuid: "246fa234db95458fa6612eca85da0521" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39099 } } peers { permanent_uuid: "74ecf34d74204b668675caf6a98b4b83" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 36925 } }
I20260808 14:32:57.693499 28981 raft_consensus.cc:374] T 1d72328d4b414860b78db7c65aa14436 P 246fa234db95458fa6612eca85da0521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:57.693499 28983 raft_consensus.cc:374] T 1d72328d4b414860b78db7c65aa14436 P 74ecf34d74204b668675caf6a98b4b83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:57.693560 28983 raft_consensus.cc:729] T 1d72328d4b414860b78db7c65aa14436 P 74ecf34d74204b668675caf6a98b4b83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74ecf34d74204b668675caf6a98b4b83, State: Initialized, Role: FOLLOWER
I20260808 14:32:57.693652 28981 raft_consensus.cc:729] T 1d72328d4b414860b78db7c65aa14436 P 246fa234db95458fa6612eca85da0521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 246fa234db95458fa6612eca85da0521, State: Initialized, Role: FOLLOWER
I20260808 14:32:57.693951 28684 heartbeater.cc:503] Master 127.7.125.126:43111 was elected leader, sending a full tablet report...
I20260808 14:32:57.694065 28983 consensus_queue.cc:260] T 1d72328d4b414860b78db7c65aa14436 P 74ecf34d74204b668675caf6a98b4b83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd12b2bc474498592078d087577541e" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43841 } } peers { permanent_uuid: "246fa234db95458fa6612eca85da0521" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39099 } } peers { permanent_uuid: "74ecf34d74204b668675caf6a98b4b83" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 36925 } }
I20260808 14:32:57.694108 28981 consensus_queue.cc:260] T 1d72328d4b414860b78db7c65aa14436 P 246fa234db95458fa6612eca85da0521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd12b2bc474498592078d087577541e" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43841 } } peers { permanent_uuid: "246fa234db95458fa6612eca85da0521" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39099 } } peers { permanent_uuid: "74ecf34d74204b668675caf6a98b4b83" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 36925 } }
I20260808 14:32:57.694281 28982 ts_tablet_manager.cc:1428] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 14:32:57.694965 28964 heartbeater.cc:503] Master 127.7.125.126:43111 was elected leader, sending a full tablet report...
I20260808 14:32:57.695050 28981 ts_tablet_manager.cc:1428] T 1d72328d4b414860b78db7c65aa14436 P 246fa234db95458fa6612eca85da0521: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260808 14:32:57.695545 28983 ts_tablet_manager.cc:1428] T 1d72328d4b414860b78db7c65aa14436 P 74ecf34d74204b668675caf6a98b4b83: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 14:32:57.696135 28822 heartbeater.cc:503] Master 127.7.125.126:43111 was elected leader, sending a full tablet report...
I20260808 14:32:57.732798 28987 raft_consensus.cc:482] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:32:57.733035 28987 raft_consensus.cc:504] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd12b2bc474498592078d087577541e" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43841 } } peers { permanent_uuid: "246fa234db95458fa6612eca85da0521" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39099 } } peers { permanent_uuid: "74ecf34d74204b668675caf6a98b4b83" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 36925 } }
I20260808 14:32:57.734364 28987 leader_election.cc:288] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 246fa234db95458fa6612eca85da0521 (127.7.125.67:39099), 74ecf34d74204b668675caf6a98b4b83 (127.7.125.66:36925)
I20260808 14:32:57.738011 28978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:57.734738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:53599 (local address 127.7.125.67:39099)
0808 14:32:57.734891 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:57.734896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:57.734915 (+ 19us) server_negotiation.cc:407] Connection header received
0808 14:32:57.735071 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:57.735077 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:57.735130 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:57.735231 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:57.735505 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.736002 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.737071 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.737459 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.737585 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:57.737635 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:57.737817 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 14:32:57.737887 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 14:32:57.738011 28686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:57.734594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.67:39099 (local address 127.7.125.65:53599)
0808 14:32:57.734772 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:57.734790 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:57.734921 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:57.735245 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:57.735248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:57.735267 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:57.735394 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:57.735400 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.736111 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.736116 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:57.736944 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:57.736954 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.737576 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.737580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:57.737679 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:57.737713 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 14:32:57.737802 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 14:32:57.737876 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 14:32:57.738399 28980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:57.735052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.125.65:36315 (local address 127.7.125.66:36925)
0808 14:32:57.735164 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:57.735169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:57.735331 (+ 162us) server_negotiation.cc:407] Connection header received
0808 14:32:57.735490 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:57.735493 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:57.735546 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:57.735633 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:32:57.735909 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.736395 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.737422 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.737807 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.737943 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:57.737989 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:57.738200 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 14:32:57.738274 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 14:32:57.738669 28914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d72328d4b414860b78db7c65aa14436" candidate_uuid: "3bd12b2bc474498592078d087577541e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "246fa234db95458fa6612eca85da0521" is_pre_election: true
I20260808 14:32:57.738817 28990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:57.734893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.66:36925 (local address 127.7.125.65:36315)
0808 14:32:57.735245 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:57.735256 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 14:32:57.735380 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:32:57.735644 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:32:57.735647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:32:57.735662 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:32:57.735795 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:57.735800 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.736504 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.736509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:57.737280 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:32:57.737288 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.737943 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.737946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:32:57.738044 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:57.738066 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 14:32:57.738627 (+ 561us) client_negotiation.cc:240] Negotiation successful
0808 14:32:57.738687 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":146,"threads_started":1}
I20260808 14:32:57.738982 28914 raft_consensus.cc:2454] T 1d72328d4b414860b78db7c65aa14436 P 246fa234db95458fa6612eca85da0521 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bd12b2bc474498592078d087577541e in term 0.
I20260808 14:32:57.739522 28772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d72328d4b414860b78db7c65aa14436" candidate_uuid: "3bd12b2bc474498592078d087577541e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74ecf34d74204b668675caf6a98b4b83" is_pre_election: true
I20260808 14:32:57.739553 28568 leader_election.cc:302] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 246fa234db95458fa6612eca85da0521, 3bd12b2bc474498592078d087577541e; no voters:
I20260808 14:32:57.739807 28772 raft_consensus.cc:2454] T 1d72328d4b414860b78db7c65aa14436 P 74ecf34d74204b668675caf6a98b4b83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bd12b2bc474498592078d087577541e in term 0.
I20260808 14:32:57.739843 28987 raft_consensus.cc:2781] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:32:57.739949 28987 raft_consensus.cc:482] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:32:57.740007 28987 raft_consensus.cc:3037] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:57.741107 28987 raft_consensus.cc:504] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd12b2bc474498592078d087577541e" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43841 } } peers { permanent_uuid: "246fa234db95458fa6612eca85da0521" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39099 } } peers { permanent_uuid: "74ecf34d74204b668675caf6a98b4b83" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 36925 } }
I20260808 14:32:57.741407 28987 leader_election.cc:288] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e [CANDIDATE]: Term 1 election: Requested vote from peers 246fa234db95458fa6612eca85da0521 (127.7.125.67:39099), 74ecf34d74204b668675caf6a98b4b83 (127.7.125.66:36925)
I20260808 14:32:57.741855 28772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d72328d4b414860b78db7c65aa14436" candidate_uuid: "3bd12b2bc474498592078d087577541e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74ecf34d74204b668675caf6a98b4b83"
I20260808 14:32:57.741864 28914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d72328d4b414860b78db7c65aa14436" candidate_uuid: "3bd12b2bc474498592078d087577541e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "246fa234db95458fa6612eca85da0521"
I20260808 14:32:57.741988 28772 raft_consensus.cc:3037] T 1d72328d4b414860b78db7c65aa14436 P 74ecf34d74204b668675caf6a98b4b83 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:57.742015 28914 raft_consensus.cc:3037] T 1d72328d4b414860b78db7c65aa14436 P 246fa234db95458fa6612eca85da0521 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:57.743407 28772 raft_consensus.cc:2454] T 1d72328d4b414860b78db7c65aa14436 P 74ecf34d74204b668675caf6a98b4b83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bd12b2bc474498592078d087577541e in term 1.
I20260808 14:32:57.743407 28914 raft_consensus.cc:2454] T 1d72328d4b414860b78db7c65aa14436 P 246fa234db95458fa6612eca85da0521 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bd12b2bc474498592078d087577541e in term 1.
I20260808 14:32:57.743724 28568 leader_election.cc:302] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bd12b2bc474498592078d087577541e, 74ecf34d74204b668675caf6a98b4b83; no voters:
I20260808 14:32:57.743920 28987 raft_consensus.cc:2781] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:32:57.744189 28987 raft_consensus.cc:686] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e [term 1 LEADER]: Becoming Leader. State: Replica: 3bd12b2bc474498592078d087577541e, State: Running, Role: LEADER
I20260808 14:32:57.744628 28987 consensus_queue.cc:237] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd12b2bc474498592078d087577541e" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43841 } } peers { permanent_uuid: "246fa234db95458fa6612eca85da0521" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39099 } } peers { permanent_uuid: "74ecf34d74204b668675caf6a98b4b83" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 36925 } }
W20260808 14:32:57.747320 28685 tablet.cc:2369] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:32:57.748113 28485 catalog_manager.cc:5495] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bd12b2bc474498592078d087577541e (127.7.125.65). New cstate: current_term: 1 leader_uuid: "3bd12b2bc474498592078d087577541e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd12b2bc474498592078d087577541e" member_type: VOTER last_known_addr { host: "127.7.125.65" port: 43841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "246fa234db95458fa6612eca85da0521" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74ecf34d74204b668675caf6a98b4b83" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 36925 } health_report { overall_health: UNKNOWN } } }
I20260808 14:32:57.772019 28977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:57.768688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50932 (local address 127.7.125.65:43841)
0808 14:32:57.768841 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:57.768845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:57.768859 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:32:57.768941 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:57.768943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:57.769003 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:57.769100 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:32:57.769317 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.769839 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.770669 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:57.770895 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:57.771149 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:57.771274 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:32:57.771689 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:32:57.771791 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:57.771840 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 14:32:57.771887 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 14:32:57.778198 28772 raft_consensus.cc:1261] T 1d72328d4b414860b78db7c65aa14436 P 74ecf34d74204b668675caf6a98b4b83 [term 1 FOLLOWER]: Refusing update from remote peer 3bd12b2bc474498592078d087577541e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:32:57.778198 28914 raft_consensus.cc:1261] T 1d72328d4b414860b78db7c65aa14436 P 246fa234db95458fa6612eca85da0521 [term 1 FOLLOWER]: Refusing update from remote peer 3bd12b2bc474498592078d087577541e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:32:57.778961 28987 consensus_queue.cc:1035] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e [LEADER]: Connected to new peer: Peer: permanent_uuid: "74ecf34d74204b668675caf6a98b4b83" member_type: VOTER last_known_addr { host: "127.7.125.66" port: 36925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:32:57.779181 28991 consensus_queue.cc:1035] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e [LEADER]: Connected to new peer: Peer: permanent_uuid: "246fa234db95458fa6612eca85da0521" member_type: VOTER last_known_addr { host: "127.7.125.67" port: 39099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:32:57.787519 28996 mvcc.cc:205] Tried to move back new op lower bound from 7316273470571360256 to 7316273470444470272. Current Snapshot: MvccSnapshot[applied={T|T < 7316273470571360256}]
I20260808 14:32:57.787550 28995 mvcc.cc:205] Tried to move back new op lower bound from 7316273470571360256 to 7316273470444470272. Current Snapshot: MvccSnapshot[applied={T|T < 7316273470571360256}]
I20260808 14:32:57.795666 28487 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:32:57.795864 28487 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:32:57.798473 28487 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5479992adddd4f659880ee20121d1b6e: Sending DeleteTablet for 3 replicas of tablet 1d72328d4b414860b78db7c65aa14436
I20260808 14:32:57.799391 28615 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d72328d4b414860b78db7c65aa14436 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:32:57 UTC) from {username='slave'} at 127.0.0.1:50928
I20260808 14:32:57.799551 28752 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d72328d4b414860b78db7c65aa14436 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:32:57 UTC) from {username='slave'} at 127.0.0.1:41340
I20260808 14:32:57.799968 29004 tablet_replica.cc:330] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e: stopping tablet replica
I20260808 14:32:57.800081 28891 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d72328d4b414860b78db7c65aa14436 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:32:57 UTC) from {username='slave'} at 127.0.0.1:41194
I20260808 14:32:57.800145 29005 tablet_replica.cc:330] T 1d72328d4b414860b78db7c65aa14436 P 74ecf34d74204b668675caf6a98b4b83: stopping tablet replica
I20260808 14:32:57.800457 29004 raft_consensus.cc:2229] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:32:57.800457 29005 raft_consensus.cc:2229] T 1d72328d4b414860b78db7c65aa14436 P 74ecf34d74204b668675caf6a98b4b83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:32:57.800796 29005 raft_consensus.cc:2258] T 1d72328d4b414860b78db7c65aa14436 P 74ecf34d74204b668675caf6a98b4b83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:32:57.801007 29004 raft_consensus.cc:2258] T 1d72328d4b414860b78db7c65aa14436 P 3bd12b2bc474498592078d087577541e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:32:57.801105 29006 tablet_replica.cc:330] T 1d72328d4b414860b78db7c65aa14436 P 246fa234db95458fa6612eca85da0521: stopping tablet replica
I20260808 14:32:57.801182 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 28549
I20260808 14:32:57.801399 29006 raft_consensus.cc:2229] T 1d72328d4b414860b78db7c65aa14436 P 246fa234db95458fa6612eca85da0521 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:32:57.801640 29006 raft_consensus.cc:2258] T 1d72328d4b414860b78db7c65aa14436 P 246fa234db95458fa6612eca85da0521 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:32:57.803757 29006 ts_tablet_manager.cc:1905] T 1d72328d4b414860b78db7c65aa14436 P 246fa234db95458fa6612eca85da0521: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:32:57.806018 29006 ts_tablet_manager.cc:1918] T 1d72328d4b414860b78db7c65aa14436 P 246fa234db95458fa6612eca85da0521: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:32:57.806142 29006 log.cc:1196] T 1d72328d4b414860b78db7c65aa14436 P 246fa234db95458fa6612eca85da0521: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-2/wal/wals/1d72328d4b414860b78db7c65aa14436
I20260808 14:32:57.806497 29006 ts_tablet_manager.cc:1939] T 1d72328d4b414860b78db7c65aa14436 P 246fa234db95458fa6612eca85da0521: Deleting consensus metadata
I20260808 14:32:57.808079 29005 ts_tablet_manager.cc:1905] T 1d72328d4b414860b78db7c65aa14436 P 74ecf34d74204b668675caf6a98b4b83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:32:57.810469 29005 ts_tablet_manager.cc:1918] T 1d72328d4b414860b78db7c65aa14436 P 74ecf34d74204b668675caf6a98b4b83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:32:57.810490 28473 catalog_manager.cc:4841] TS 246fa234db95458fa6612eca85da0521 (127.7.125.67:39099): tablet 1d72328d4b414860b78db7c65aa14436 (table test-table [id=53aa7f8ef1aa459a98aacd1d185be98d]) successfully deleted
I20260808 14:32:57.810606 29005 log.cc:1196] T 1d72328d4b414860b78db7c65aa14436 P 74ecf34d74204b668675caf6a98b4b83: Deleting WAL directory at /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199316863421-7669-0/minicluster-data/ts-1/wal/wals/1d72328d4b414860b78db7c65aa14436
I20260808 14:32:57.811012 29005 ts_tablet_manager.cc:1939] T 1d72328d4b414860b78db7c65aa14436 P 74ecf34d74204b668675caf6a98b4b83: Deleting consensus metadata
I20260808 14:32:57.812289 28473 catalog_manager.cc:4841] TS 74ecf34d74204b668675caf6a98b4b83 (127.7.125.66:36925): tablet 1d72328d4b414860b78db7c65aa14436 (table test-table [id=53aa7f8ef1aa459a98aacd1d185be98d]) successfully deleted
W20260808 14:32:57.813079 28471 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.125.65:43841 (error 108)
I20260808 14:32:57.813319 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 28687
I20260808 14:32:57.816197 28975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:57.815803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.65:43841 (local address 127.0.0.1:50936)
0808 14:32:57.815987 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 14:32:57.816083 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.65:43841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260808 14:32:57.816408 28471 catalog_manager.cc:4584] TS 3bd12b2bc474498592078d087577541e (127.7.125.65:43841): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1d72328d4b414860b78db7c65aa14436: Network error: Client connection negotiation failed: client connection to 127.7.125.65:43841: connect: Connection refused (error 111)
I20260808 14:32:57.821664 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 28826
I20260808 14:32:57.828775 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 28454
2026-08-08T14:32:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1054 ms)
[----------] 6 tests from EncryptionPolicyTest (11012 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:32:57 dist-test-slave-w206 krb5kdc[29010](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:32:57 dist-test-slave-w206 krb5kdc[29010](info): set up 2 sockets
Aug 08 14:32:57 dist-test-slave-w206 krb5kdc[29010](info): commencing operation
krb5kdc: starting...
W20260808 14:32:58.912999 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:32:59 dist-test-slave-w206 krb5kdc[29010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:32:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:32:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:32:59.054929 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42561
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42561
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:32:59.169617 29024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:32:59.169939 29024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:32:59.169984 29024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:32:59.174290 29024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:32:59.174372 29024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:32:59.174393 29024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:32:59.174412 29024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:32:59.174427 29024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:32:59.179984 29024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42561
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:32:59.181088 29024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:32:59.182220 29024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:32:59.191449 29030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:59.191447 29032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:32:59.191439 29029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:32:59.191740 29024 server_base.cc:1034] running on GCE node
I20260808 14:32:59.192479 29024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:32:59.193760 29024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:32:59.195055 29024 hybrid_clock.cc:648] HybridClock initialized: now 1786199579195027 us; error 38 us; skew 500 ppm
Aug 08 14:32:59 dist-test-slave-w206 krb5kdc[29010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199579, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:32:59.203426 29024 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:32:59.204537 29024 webserver.cc:460] Webserver started at http://127.0.0.1:36981/ using document root <none> and password file <none>
I20260808 14:32:59.205158 29024 fs_manager.cc:359] Metadata directory not provided
I20260808 14:32:59.205258 29024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:32:59.205511 29024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:32:59.207782 29024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "dd461da873c24a9691c5d015aa6d9824"
format_stamp: "Formatted at 2026-08-08 14:32:59 on dist-test-slave-w206"
server_key: "4d940bbed31986a345e621e44f21d3bd"
server_key_iv: "e2af6189a07a6bca28f0e17804bb8d1b"
server_key_version: "encryptionkey@0"
I20260808 14:32:59.208379 29024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "dd461da873c24a9691c5d015aa6d9824"
format_stamp: "Formatted at 2026-08-08 14:32:59 on dist-test-slave-w206"
server_key: "4d940bbed31986a345e621e44f21d3bd"
server_key_iv: "e2af6189a07a6bca28f0e17804bb8d1b"
server_key_version: "encryptionkey@0"
I20260808 14:32:59.212467 29024 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:32:59.215183 29039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:32:59.216571 29024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:32:59.216744 29024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "dd461da873c24a9691c5d015aa6d9824"
format_stamp: "Formatted at 2026-08-08 14:32:59 on dist-test-slave-w206"
server_key: "4d940bbed31986a345e621e44f21d3bd"
server_key_iv: "e2af6189a07a6bca28f0e17804bb8d1b"
server_key_version: "encryptionkey@0"
I20260808 14:32:59.216877 29024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:32:59.250357 29024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:32:59.253428 29024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:32:59.253641 29024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:32:59.261338 29024 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42561
I20260808 14:32:59.261365 29098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42561 every 8 connection(s)
I20260808 14:32:59.262275 29024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:32:59.263095 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 29024
I20260808 14:32:59.263237 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:32:59.263561 7669 external_mini_cluster.cc:1428] Setting key 67be2194f933ac896fcc0bce650bf997
I20260808 14:32:59.265535 29099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:32:59.271247 29099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824: Bootstrap starting.
Aug 08 14:32:59 dist-test-slave-w206 krb5kdc[29010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 14:32:59.273798 29099 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824: Neither blocks nor log segments found. Creating new log.
I20260808 14:32:59.274698 29099 log.cc:824] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824: Log is configured to *not* fsync() on all Append() calls
I20260808 14:32:59.277189 29099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824: No bootstrap required, opened a new log
I20260808 14:32:59.278344 29102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:59.265044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43759 (local address 127.0.0.1:42561)
0808 14:32:59.265513 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:59.265527 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:59.265561 (+ 34us) server_negotiation.cc:407] Connection header received
0808 14:32:59.266255 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:59.266272 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:59.266601 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:59.266943 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:59.267800 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:59.268788 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:59.269656 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:59.270122 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:59.270294 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:59.273231 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:59.273236 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:59.273249 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:59.273276 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:59.275301 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:59.275982 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:59.275987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:59.275991 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:59.276096 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:59.276444 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:59.276446 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:59.276447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:59.276828 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:59.276986 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:59.277367 (+ 381us) server_negotiation.cc:299] Negotiation successful
0808 14:32:59.277627 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":165,"threads_started":1}
I20260808 14:32:59.281468 29099 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd461da873c24a9691c5d015aa6d9824" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42561 } }
I20260808 14:32:59.281749 29099 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:32:59.281769 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:32:59.281821 29099 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd461da873c24a9691c5d015aa6d9824, State: Initialized, Role: FOLLOWER
I20260808 14:32:59.282311 29099 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd461da873c24a9691c5d015aa6d9824" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42561 } }
I20260808 14:32:59.282503 29099 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:32:59.282569 29099 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:32:59.282666 29099 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:32:59.284030 29099 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd461da873c24a9691c5d015aa6d9824" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42561 } }
I20260808 14:32:59.284410 29099 leader_election.cc:302] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd461da873c24a9691c5d015aa6d9824; no voters:
I20260808 14:32:59.284780 29099 leader_election.cc:288] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:32:59.284870 29109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:32:59.285115 29109 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824 [term 1 LEADER]: Becoming Leader. State: Replica: dd461da873c24a9691c5d015aa6d9824, State: Running, Role: LEADER
I20260808 14:32:59.285539 29109 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd461da873c24a9691c5d015aa6d9824" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42561 } }
I20260808 14:32:59.286266 29099 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:32:59.286770 29109 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd461da873c24a9691c5d015aa6d9824. Latest consensus state: current_term: 1 leader_uuid: "dd461da873c24a9691c5d015aa6d9824" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd461da873c24a9691c5d015aa6d9824" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42561 } } }
I20260808 14:32:59.286906 29109 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824 [sys.catalog]: This master's current role is: LEADER
I20260808 14:32:59.287396 29110 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd461da873c24a9691c5d015aa6d9824" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd461da873c24a9691c5d015aa6d9824" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42561 } } }
I20260808 14:32:59.287528 29110 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824 [sys.catalog]: This master's current role is: LEADER
W20260808 14:32:59.291563 29124 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:32:59.291700 29124 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:32:59.291846 29115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:32:59.292601 29115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:32:59.294411 29102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:32:59.284748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46081 (local address 127.0.0.1:42561)
0808 14:32:59.284862 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 14:32:59.284867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:32:59.284885 (+ 18us) server_negotiation.cc:407] Connection header received
0808 14:32:59.285074 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:32:59.285077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:32:59.285125 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:32:59.285210 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:32:59.286142 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:59.287037 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:59.288288 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:32:59.288618 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:32:59.288732 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:32:59.290528 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:32:59.290533 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:32:59.290536 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:32:59.290562 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:32:59.292369 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:59.292942 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:59.292945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:59.292947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:59.293039 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:32:59.293433 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:32:59.293436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:32:59.293438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:32:59.293663 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:32:59.293827 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 14:32:59.294148 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 14:32:59.294252 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 14:32:59.298430 29115 catalog_manager.cc:1347] Generated new cluster ID: 95d0f263b75c4431b7ba9b602b5665a5
I20260808 14:32:59.298831 29115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:32:59.316232 29115 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:32:59.317255 29115 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:32:59.323704 29115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd461da873c24a9691c5d015aa6d9824: Generated new TSK 0
I20260808 14:32:59.324574 29115 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:32:59.339366 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 29024
W20260808 14:32:59.347234 29008 connection.cc:535] client connection to 127.0.0.1:42561 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42561): Connection reset by peer (error 104)
2026-08-08T14:32:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1510 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:32:59 dist-test-slave-w206 krb5kdc[29130](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:32:59 dist-test-slave-w206 krb5kdc[29130](info): set up 2 sockets
Aug 08 14:32:59 dist-test-slave-w206 krb5kdc[29130](info): commencing operation
krb5kdc: starting...
W20260808 14:33:00.416224 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:33:00 dist-test-slave-w206 krb5kdc[29130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:33:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:33:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:33:00.549702 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36273
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:33:00.666791 29143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:33:00.667079 29143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:33:00.667166 29143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:33:00.671253 29143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:33:00.671317 29143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:33:00.671337 29143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:33:00.671393 29143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:33:00.671423 29143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:33:00.676865 29143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36273
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:33:00.677973 29143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:33:00.679057 29143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:33:00.687769 29148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:00.687769 29151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:00.687915 29143 server_base.cc:1034] running on GCE node
W20260808 14:33:00.687795 29149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:00.688633 29143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:33:00.689877 29143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:33:00.691099 29143 hybrid_clock.cc:648] HybridClock initialized: now 1786199580691066 us; error 48 us; skew 500 ppm
Aug 08 14:33:00 dist-test-slave-w206 krb5kdc[29130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199580, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:33:00.693921 29143 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:33:00.694830 29143 webserver.cc:460] Webserver started at http://127.0.0.1:36277/ using document root <none> and password file <none>
I20260808 14:33:00.695463 29143 fs_manager.cc:359] Metadata directory not provided
I20260808 14:33:00.695561 29143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:33:00.695796 29143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:33:00.697978 29143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "d1c737437c7a4a919cc986422cd770ed"
format_stamp: "Formatted at 2026-08-08 14:33:00 on dist-test-slave-w206"
server_key: "99f7a1c76c274358dabf818cd8a0d597"
server_key_iv: "20fa069f2f999b3bb74b6261233745af"
server_key_version: "encryptionkey@0"
I20260808 14:33:00.698568 29143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "d1c737437c7a4a919cc986422cd770ed"
format_stamp: "Formatted at 2026-08-08 14:33:00 on dist-test-slave-w206"
server_key: "99f7a1c76c274358dabf818cd8a0d597"
server_key_iv: "20fa069f2f999b3bb74b6261233745af"
server_key_version: "encryptionkey@0"
I20260808 14:33:00.702490 29143 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:33:00.705103 29159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:33:00.706442 29143 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 14:33:00.706580 29143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "d1c737437c7a4a919cc986422cd770ed"
format_stamp: "Formatted at 2026-08-08 14:33:00 on dist-test-slave-w206"
server_key: "99f7a1c76c274358dabf818cd8a0d597"
server_key_iv: "20fa069f2f999b3bb74b6261233745af"
server_key_version: "encryptionkey@0"
I20260808 14:33:00.706700 29143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:33:00.723244 29143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:33:00.726451 29143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:33:00.726683 29143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:33:00.733932 29143 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36273
I20260808 14:33:00.734006 29217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36273 every 8 connection(s)
I20260808 14:33:00.734943 29143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:33:00.737097 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 29143
I20260808 14:33:00.737237 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:33:00.737573 7669 external_mini_cluster.cc:1428] Setting key b3dd8bed460d6972f095aba6f28affbd
I20260808 14:33:00.738158 29218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:33:00.744313 29218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1c737437c7a4a919cc986422cd770ed: Bootstrap starting.
I20260808 14:33:00.746383 29221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:00.738948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36999 (local address 127.0.0.1:36273)
0808 14:33:00.739413 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:00.739430 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:00.739466 (+ 36us) server_negotiation.cc:407] Connection header received
0808 14:33:00.740131 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:00.740148 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:00.740482 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:00.740813 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:00.741219 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:00.742187 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:00.743123 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:00.743609 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:00.743783 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:00.744782 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:00.744787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:00.744800 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:00.745173 (+ 373us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:00.745226 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:00.745247 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:00.745400 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:00.745547 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 14:33:00.745689 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":142,"threads_started":1}
I20260808 14:33:00.746644 29218 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1c737437c7a4a919cc986422cd770ed: Neither blocks nor log segments found. Creating new log.
I20260808 14:33:00.747459 29218 log.cc:824] T 00000000000000000000000000000000 P d1c737437c7a4a919cc986422cd770ed: Log is configured to *not* fsync() on all Append() calls
I20260808 14:33:00.749281 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:33:00.749480 29218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1c737437c7a4a919cc986422cd770ed: No bootstrap required, opened a new log
I20260808 14:33:00.754256 29218 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1c737437c7a4a919cc986422cd770ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1c737437c7a4a919cc986422cd770ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36273 } }
I20260808 14:33:00.754562 29218 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1c737437c7a4a919cc986422cd770ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:33:00.754657 29218 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1c737437c7a4a919cc986422cd770ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c737437c7a4a919cc986422cd770ed, State: Initialized, Role: FOLLOWER
I20260808 14:33:00.755168 29218 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1c737437c7a4a919cc986422cd770ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1c737437c7a4a919cc986422cd770ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36273 } }
I20260808 14:33:00.755347 29218 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1c737437c7a4a919cc986422cd770ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:33:00.755419 29218 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1c737437c7a4a919cc986422cd770ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:33:00.755538 29218 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1c737437c7a4a919cc986422cd770ed [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:33:00.755903 29221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:00.751629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47275 (local address 127.0.0.1:36273)
0808 14:33:00.751771 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:00.751775 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:00.751886 (+ 111us) server_negotiation.cc:407] Connection header received
0808 14:33:00.752007 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:00.752011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:00.752065 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:00.752183 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:00.752381 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:00.753205 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:00.753992 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:00.754329 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:00.754438 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:00.755345 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:00.755348 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:00.755353 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:00.755369 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:00.755399 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:00.755412 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:00.755508 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:00.755716 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 14:33:00.755757 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 14:33:00.756749 29218 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1c737437c7a4a919cc986422cd770ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1c737437c7a4a919cc986422cd770ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36273 } }
I20260808 14:33:00.757133 29218 leader_election.cc:302] T 00000000000000000000000000000000 P d1c737437c7a4a919cc986422cd770ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1c737437c7a4a919cc986422cd770ed; no voters:
I20260808 14:33:00.757457 29218 leader_election.cc:288] T 00000000000000000000000000000000 P d1c737437c7a4a919cc986422cd770ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:33:00.757544 29228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1c737437c7a4a919cc986422cd770ed [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:33:00.757788 29228 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1c737437c7a4a919cc986422cd770ed [term 1 LEADER]: Becoming Leader. State: Replica: d1c737437c7a4a919cc986422cd770ed, State: Running, Role: LEADER
I20260808 14:33:00.758086 29228 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1c737437c7a4a919cc986422cd770ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1c737437c7a4a919cc986422cd770ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36273 } }
I20260808 14:33:00.758740 29218 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1c737437c7a4a919cc986422cd770ed [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:33:00.759980 29230 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1c737437c7a4a919cc986422cd770ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1c737437c7a4a919cc986422cd770ed. Latest consensus state: current_term: 1 leader_uuid: "d1c737437c7a4a919cc986422cd770ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1c737437c7a4a919cc986422cd770ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36273 } } }
I20260808 14:33:00.760016 29229 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1c737437c7a4a919cc986422cd770ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1c737437c7a4a919cc986422cd770ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1c737437c7a4a919cc986422cd770ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36273 } } }
I20260808 14:33:00.760166 29230 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1c737437c7a4a919cc986422cd770ed [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:00.760198 29229 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1c737437c7a4a919cc986422cd770ed [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:00.760597 29237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:33:00.763705 29237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:33:00.768980 29237 catalog_manager.cc:1347] Generated new cluster ID: 2ee39f6fa20d4eceb1104e8d203757ef
I20260808 14:33:00.769049 29237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:33:00.775725 29237 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:33:00.776937 29237 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:33:00.790436 29237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1c737437c7a4a919cc986422cd770ed: Generated new TSK 0
I20260808 14:33:00.791365 29237 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:33:00.796411 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 29143
W20260808 14:33:00.804098 29128 connection.cc:535] client connection to 127.0.0.1:36273 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36273): Connection reset by peer (error 104)
2026-08-08T14:33:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1456 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:33:00 dist-test-slave-w206 krb5kdc[29253](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:33:00 dist-test-slave-w206 krb5kdc[29253](info): set up 2 sockets
Aug 08 14:33:00 dist-test-slave-w206 krb5kdc[29253](info): commencing operation
krb5kdc: starting...
W20260808 14:33:01.884357 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:33:01 dist-test-slave-w206 krb5kdc[29253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:33:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:33:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:33:02.021641 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35011
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:33:02.138458 29266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:33:02.138758 29266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:33:02.138803 29266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:33:02.144513 29266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:33:02.144642 29266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:33:02.144678 29266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:33:02.144732 29266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:33:02.144771 29266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:33:02.153010 29266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35011
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:33:02.154150 29266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:33:02.155259 29266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:33:02.164583 29272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:02.164599 29273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:02.164593 29275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:02.165005 29266 server_base.cc:1034] running on GCE node
I20260808 14:33:02.165589 29266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:33:02.166738 29266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:33:02.167934 29266 hybrid_clock.cc:648] HybridClock initialized: now 1786199582167897 us; error 55 us; skew 500 ppm
Aug 08 14:33:02 dist-test-slave-w206 krb5kdc[29253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199582, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:33:02.170872 29266 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:33:02.172024 29266 webserver.cc:460] Webserver started at http://127.0.0.1:33253/ using document root <none> and password file <none>
I20260808 14:33:02.172603 29266 fs_manager.cc:359] Metadata directory not provided
I20260808 14:33:02.172669 29266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:33:02.172851 29266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:33:02.175107 29266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "acf0dc870c2e486ebd7bce50d3a5e764"
format_stamp: "Formatted at 2026-08-08 14:33:02 on dist-test-slave-w206"
server_key: "5614d28408a58e7bbedb85a319434b9e"
server_key_iv: "772adc5a813dd38c4976ec627626f701"
server_key_version: "encryptionkey@0"
I20260808 14:33:02.175670 29266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "acf0dc870c2e486ebd7bce50d3a5e764"
format_stamp: "Formatted at 2026-08-08 14:33:02 on dist-test-slave-w206"
server_key: "5614d28408a58e7bbedb85a319434b9e"
server_key_iv: "772adc5a813dd38c4976ec627626f701"
server_key_version: "encryptionkey@0"
I20260808 14:33:02.179760 29266 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 14:33:02.182523 29284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:33:02.183941 29266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:33:02.184118 29266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "acf0dc870c2e486ebd7bce50d3a5e764"
format_stamp: "Formatted at 2026-08-08 14:33:02 on dist-test-slave-w206"
server_key: "5614d28408a58e7bbedb85a319434b9e"
server_key_iv: "772adc5a813dd38c4976ec627626f701"
server_key_version: "encryptionkey@0"
I20260808 14:33:02.184257 29266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:33:02.189280 29266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:33:02.192729 29266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:33:02.192951 29266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:33:02.200649 29266 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35011
I20260808 14:33:02.200668 29342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35011 every 8 connection(s)
I20260808 14:33:02.201648 29266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:33:02.204571 29343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:33:02.209510 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 29266
I20260808 14:33:02.209673 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:33:02.210011 7669 external_mini_cluster.cc:1428] Setting key 7c3ef8ae228fa45194f1af89336961b4
I20260808 14:33:02.210857 29343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acf0dc870c2e486ebd7bce50d3a5e764: Bootstrap starting.
I20260808 14:33:02.213512 29343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acf0dc870c2e486ebd7bce50d3a5e764: Neither blocks nor log segments found. Creating new log.
I20260808 14:33:02.214531 29343 log.cc:824] T 00000000000000000000000000000000 P acf0dc870c2e486ebd7bce50d3a5e764: Log is configured to *not* fsync() on all Append() calls
I20260808 14:33:02.215787 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:02.211513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38743 (local address 127.0.0.1:35011)
0808 14:33:02.212014 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:02.212034 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:02.212072 (+ 38us) server_negotiation.cc:407] Connection header received
0808 14:33:02.212742 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:02.212760 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:02.213087 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:02.213396 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:02.214394 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:02.214400 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:02.214416 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:02.214427 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:02.214482 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:02.214502 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:02.214593 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:02.214783 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 14:33:02.215041 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":162,"threads_started":1}
I20260808 14:33:02.216701 29343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acf0dc870c2e486ebd7bce50d3a5e764: No bootstrap required, opened a new log
I20260808 14:33:02.218783 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:33:02.221350 29343 raft_consensus.cc:348] T 00000000000000000000000000000000 P acf0dc870c2e486ebd7bce50d3a5e764 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf0dc870c2e486ebd7bce50d3a5e764" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35011 } }
I20260808 14:33:02.221674 29343 raft_consensus.cc:374] T 00000000000000000000000000000000 P acf0dc870c2e486ebd7bce50d3a5e764 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:33:02.221743 29343 raft_consensus.cc:729] T 00000000000000000000000000000000 P acf0dc870c2e486ebd7bce50d3a5e764 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf0dc870c2e486ebd7bce50d3a5e764, State: Initialized, Role: FOLLOWER
I20260808 14:33:02.222240 29343 consensus_queue.cc:260] T 00000000000000000000000000000000 P acf0dc870c2e486ebd7bce50d3a5e764 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf0dc870c2e486ebd7bce50d3a5e764" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35011 } }
I20260808 14:33:02.222400 29343 raft_consensus.cc:388] T 00000000000000000000000000000000 P acf0dc870c2e486ebd7bce50d3a5e764 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:33:02.222472 29343 raft_consensus.cc:482] T 00000000000000000000000000000000 P acf0dc870c2e486ebd7bce50d3a5e764 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:33:02.222605 29343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acf0dc870c2e486ebd7bce50d3a5e764 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:33:02.223526 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:02.221504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56123 (local address 127.0.0.1:35011)
0808 14:33:02.221658 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:02.221663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:02.221934 (+ 271us) server_negotiation.cc:407] Connection header received
0808 14:33:02.222082 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:02.222086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:02.222129 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:02.222228 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:02.223101 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:02.223105 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:02.223108 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:02.223121 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:02.223150 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:02.223169 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:02.223274 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:02.223354 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 14:33:02.223396 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 14:33:02.223871 29343 raft_consensus.cc:504] T 00000000000000000000000000000000 P acf0dc870c2e486ebd7bce50d3a5e764 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf0dc870c2e486ebd7bce50d3a5e764" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35011 } }
I20260808 14:33:02.224241 29343 leader_election.cc:302] T 00000000000000000000000000000000 P acf0dc870c2e486ebd7bce50d3a5e764 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: acf0dc870c2e486ebd7bce50d3a5e764; no voters:
I20260808 14:33:02.224618 29343 leader_election.cc:288] T 00000000000000000000000000000000 P acf0dc870c2e486ebd7bce50d3a5e764 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:33:02.224747 29353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P acf0dc870c2e486ebd7bce50d3a5e764 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:33:02.225034 29353 raft_consensus.cc:686] T 00000000000000000000000000000000 P acf0dc870c2e486ebd7bce50d3a5e764 [term 1 LEADER]: Becoming Leader. State: Replica: acf0dc870c2e486ebd7bce50d3a5e764, State: Running, Role: LEADER
I20260808 14:33:02.225374 29353 consensus_queue.cc:237] T 00000000000000000000000000000000 P acf0dc870c2e486ebd7bce50d3a5e764 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf0dc870c2e486ebd7bce50d3a5e764" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35011 } }
I20260808 14:33:02.226068 29343 sys_catalog.cc:564] T 00000000000000000000000000000000 P acf0dc870c2e486ebd7bce50d3a5e764 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:33:02.227191 29354 sys_catalog.cc:455] T 00000000000000000000000000000000 P acf0dc870c2e486ebd7bce50d3a5e764 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "acf0dc870c2e486ebd7bce50d3a5e764" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf0dc870c2e486ebd7bce50d3a5e764" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35011 } } }
I20260808 14:33:02.227249 29355 sys_catalog.cc:455] T 00000000000000000000000000000000 P acf0dc870c2e486ebd7bce50d3a5e764 [sys.catalog]: SysCatalogTable state changed. Reason: New leader acf0dc870c2e486ebd7bce50d3a5e764. Latest consensus state: current_term: 1 leader_uuid: "acf0dc870c2e486ebd7bce50d3a5e764" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf0dc870c2e486ebd7bce50d3a5e764" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35011 } } }
I20260808 14:33:02.227334 29354 sys_catalog.cc:458] T 00000000000000000000000000000000 P acf0dc870c2e486ebd7bce50d3a5e764 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:02.227380 29355 sys_catalog.cc:458] T 00000000000000000000000000000000 P acf0dc870c2e486ebd7bce50d3a5e764 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:02.227692 29364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:33:02.231232 29364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:33:02.238695 29364 catalog_manager.cc:1347] Generated new cluster ID: ee9097312ef1423fa1dee3460ff07c19
I20260808 14:33:02.238793 29364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:33:02.262662 29364 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:33:02.263080 29364 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:33:02.272719 29364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P acf0dc870c2e486ebd7bce50d3a5e764: Generated new TSK 0
I20260808 14:33:02.273653 29364 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:33:02.285472 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 29266
2026-08-08T14:33:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1488 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:33:02 dist-test-slave-w206 krb5kdc[29376](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:33:02 dist-test-slave-w206 krb5kdc[29376](info): set up 2 sockets
Aug 08 14:33:02 dist-test-slave-w206 krb5kdc[29376](info): commencing operation
krb5kdc: starting...
W20260808 14:33:03.352671 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:33:03 dist-test-slave-w206 krb5kdc[29376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:33:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:33:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:33:03.486689 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33739
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:33:03.600353 29390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:33:03.600713 29390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:33:03.600804 29390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:33:03.604915 29390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:33:03.605026 29390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:33:03.605062 29390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:33:03.605103 29390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:33:03.605139 29390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:33:03.610636 29390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:33:03.611931 29390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:33:03.613040 29390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:33:03.621860 29401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:03.621860 29398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:03.621881 29399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:03.622079 29390 server_base.cc:1034] running on GCE node
I20260808 14:33:03.622797 29390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:33:03.623994 29390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:33:03.625188 29390 hybrid_clock.cc:648] HybridClock initialized: now 1786199583625159 us; error 42 us; skew 500 ppm
Aug 08 14:33:03 dist-test-slave-w206 krb5kdc[29376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199583, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:33:03.628124 29390 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:33:03.629060 29390 webserver.cc:460] Webserver started at http://127.0.0.1:45229/ using document root <none> and password file <none>
I20260808 14:33:03.629590 29390 fs_manager.cc:359] Metadata directory not provided
I20260808 14:33:03.629649 29390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:33:03.629817 29390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:33:03.631917 29390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "0d7148f5acc34ad2a9b771d244f99f73"
format_stamp: "Formatted at 2026-08-08 14:33:03 on dist-test-slave-w206"
server_key: "742a308a25e7cce3a8a77caa40ab2cbe"
server_key_iv: "48f4a4deab547df54fa8a40c6b602083"
server_key_version: "encryptionkey@0"
I20260808 14:33:03.632452 29390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "0d7148f5acc34ad2a9b771d244f99f73"
format_stamp: "Formatted at 2026-08-08 14:33:03 on dist-test-slave-w206"
server_key: "742a308a25e7cce3a8a77caa40ab2cbe"
server_key_iv: "48f4a4deab547df54fa8a40c6b602083"
server_key_version: "encryptionkey@0"
I20260808 14:33:03.636270 29390 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:33:03.638834 29408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:33:03.640130 29390 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 14:33:03.640239 29390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "0d7148f5acc34ad2a9b771d244f99f73"
format_stamp: "Formatted at 2026-08-08 14:33:03 on dist-test-slave-w206"
server_key: "742a308a25e7cce3a8a77caa40ab2cbe"
server_key_iv: "48f4a4deab547df54fa8a40c6b602083"
server_key_version: "encryptionkey@0"
I20260808 14:33:03.640332 29390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:33:03.655706 29390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:33:03.658716 29390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:33:03.658870 29390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:33:03.666417 29390 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33739
I20260808 14:33:03.666435 29464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33739 every 8 connection(s)
I20260808 14:33:03.667452 29390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:33:03.670102 29465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:33:03.674089 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 29390
I20260808 14:33:03.674258 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:33:03.674553 7669 external_mini_cluster.cc:1428] Setting key 5e001aa00fcde6c9828d56806a810694
I20260808 14:33:03.676170 29465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73: Bootstrap starting.
I20260808 14:33:03.678417 29465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73: Neither blocks nor log segments found. Creating new log.
I20260808 14:33:03.679230 29465 log.cc:824] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73: Log is configured to *not* fsync() on all Append() calls
I20260808 14:33:03.681185 29465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73: No bootstrap required, opened a new log
Aug 08 14:33:03 dist-test-slave-w206 krb5kdc[29376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 14:33:03.685144 29465 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d7148f5acc34ad2a9b771d244f99f73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33739 } }
I20260808 14:33:03.685443 29465 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:33:03.685525 29465 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d7148f5acc34ad2a9b771d244f99f73, State: Initialized, Role: FOLLOWER
I20260808 14:33:03.686043 29465 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d7148f5acc34ad2a9b771d244f99f73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33739 } }
I20260808 14:33:03.686173 29465 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:33:03.686261 29465 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:33:03.686379 29465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:33:03.687384 29465 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d7148f5acc34ad2a9b771d244f99f73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33739 } }
I20260808 14:33:03.687727 29465 leader_election.cc:302] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d7148f5acc34ad2a9b771d244f99f73; no voters:
I20260808 14:33:03.688079 29465 leader_election.cc:288] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:33:03.688256 29470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:33:03.688495 29470 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73 [term 1 LEADER]: Becoming Leader. State: Replica: 0d7148f5acc34ad2a9b771d244f99f73, State: Running, Role: LEADER
I20260808 14:33:03.688834 29470 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d7148f5acc34ad2a9b771d244f99f73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33739 } }
I20260808 14:33:03.689414 29465 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:33:03.689728 29468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:03.676049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52355 (local address 127.0.0.1:33739)
0808 14:33:03.676516 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:03.676532 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:03.676570 (+ 38us) server_negotiation.cc:407] Connection header received
0808 14:33:03.677231 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:03.677248 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:03.677590 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:03.677948 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:03.678848 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:03.680207 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:03.681078 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:03.681611 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:03.681790 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:03.684425 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:03.684431 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:03.684446 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:03.684472 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:03.686521 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:03.687008 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:03.687015 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:03.687020 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:03.687144 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:03.687445 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:03.687448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:03.687450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:03.687952 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:03.688163 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:03.688485 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 14:33:03.688818 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":147,"threads_started":1}
I20260808 14:33:03.690016 29470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d7148f5acc34ad2a9b771d244f99f73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d7148f5acc34ad2a9b771d244f99f73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33739 } } }
I20260808 14:33:03.690145 29470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:03.690383 29470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d7148f5acc34ad2a9b771d244f99f73. Latest consensus state: current_term: 1 leader_uuid: "0d7148f5acc34ad2a9b771d244f99f73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d7148f5acc34ad2a9b771d244f99f73" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33739 } } }
I20260808 14:33:03.690478 29470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:03.693053 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 14:33:03.695056 29485 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:33:03.695142 29485 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:33:03.695202 29476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:33:03.696082 29476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:33:03.702852 29476 catalog_manager.cc:1347] Generated new cluster ID: 9affc780c48443708a3c0272b9db01cf
I20260808 14:33:03.702941 29476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:33:03.704753 29468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:03.695421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36309 (local address 127.0.0.1:33739)
0808 14:33:03.695566 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:03.695572 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:03.695844 (+ 272us) server_negotiation.cc:407] Connection header received
0808 14:33:03.696000 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:03.696003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:03.696053 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:03.696139 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:03.696867 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:03.697400 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:03.698353 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:03.698733 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:03.698849 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:03.700582 (+ 1733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:03.700586 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:03.700589 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:03.700620 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:03.702397 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:03.702929 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:03.702933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:03.702935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:03.703050 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:03.703347 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:03.703349 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:03.703350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:03.703556 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:03.704344 (+ 788us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:03.704434 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 14:33:03.704570 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:33:03.724112 29476 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:33:03.725126 29476 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:33:03.733183 29476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d7148f5acc34ad2a9b771d244f99f73: Generated new TSK 0
I20260808 14:33:03.733979 29476 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:33:03.740597 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 29390
2026-08-08T14:33:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1455 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:33:03 dist-test-slave-w206 krb5kdc[29496](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:33:03 dist-test-slave-w206 krb5kdc[29496](info): set up 2 sockets
Aug 08 14:33:03 dist-test-slave-w206 krb5kdc[29496](info): commencing operation
krb5kdc: starting...
W20260808 14:33:04.831699 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:33:04 dist-test-slave-w206 krb5kdc[29496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:33:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:33:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:33:04.966150 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45233
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:33:05.081754 29510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:33:05.082065 29510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:33:05.082160 29510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:33:05.086210 29510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:33:05.086303 29510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:33:05.086349 29510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:33:05.086407 29510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:33:05.086448 29510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:33:05.091878 29510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:33:05.093014 29510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:33:05.094099 29510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:33:05.102885 29520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:05.102892 29516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:05.102893 29518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:05.103184 29510 server_base.cc:1034] running on GCE node
I20260808 14:33:05.103897 29510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:33:05.105062 29510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:33:05.106259 29510 hybrid_clock.cc:648] HybridClock initialized: now 1786199585106248 us; error 33 us; skew 500 ppm
Aug 08 14:33:05 dist-test-slave-w206 krb5kdc[29496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199585, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:33:05.116339 29510 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:33:05.117446 29510 webserver.cc:460] Webserver started at http://127.0.0.1:38239/ using document root <none> and password file <none>
I20260808 14:33:05.118065 29510 fs_manager.cc:359] Metadata directory not provided
I20260808 14:33:05.118165 29510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:33:05.118422 29510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:33:05.120682 29510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "66dbdcada2ec4ec1adaec4ac14bbe368"
format_stamp: "Formatted at 2026-08-08 14:33:05 on dist-test-slave-w206"
server_key: "2f3954150dc26d712c3b474ce766c959"
server_key_iv: "1e9c90f6206ab9eb48006f8480f20d2d"
server_key_version: "encryptionkey@0"
I20260808 14:33:05.121272 29510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "66dbdcada2ec4ec1adaec4ac14bbe368"
format_stamp: "Formatted at 2026-08-08 14:33:05 on dist-test-slave-w206"
server_key: "2f3954150dc26d712c3b474ce766c959"
server_key_iv: "1e9c90f6206ab9eb48006f8480f20d2d"
server_key_version: "encryptionkey@0"
I20260808 14:33:05.125111 29510 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260808 14:33:05.127655 29528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:33:05.128868 29510 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:33:05.129012 29510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "66dbdcada2ec4ec1adaec4ac14bbe368"
format_stamp: "Formatted at 2026-08-08 14:33:05 on dist-test-slave-w206"
server_key: "2f3954150dc26d712c3b474ce766c959"
server_key_iv: "1e9c90f6206ab9eb48006f8480f20d2d"
server_key_version: "encryptionkey@0"
I20260808 14:33:05.129124 29510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:33:05.141269 29510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:33:05.144760 29510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:33:05.144979 29510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:33:05.152760 29510 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45233
I20260808 14:33:05.152781 29584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45233 every 8 connection(s)
I20260808 14:33:05.153789 29510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:33:05.154213 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 29510
I20260808 14:33:05.154381 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:33:05.154680 7669 external_mini_cluster.cc:1428] Setting key 05137e3f27e8475b06116d66cd4ce373
I20260808 14:33:05.156948 29585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:33:05.164036 29585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66dbdcada2ec4ec1adaec4ac14bbe368: Bootstrap starting.
I20260808 14:33:05.164453 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:05.156186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44611 (local address 127.0.0.1:45233)
0808 14:33:05.156603 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:05.156620 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:05.156657 (+ 37us) server_negotiation.cc:407] Connection header received
0808 14:33:05.157358 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:05.157376 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:05.157726 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:05.158014 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:05.158438 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:05.159594 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:05.160900 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:05.161404 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:05.161523 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:05.162484 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:05.162489 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:05.162502 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:05.162993 (+ 491us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:05.163084 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:05.163110 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:05.163189 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:05.163365 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 14:33:05.163607 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":148,"threads_started":1}
I20260808 14:33:05.166622 29585 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66dbdcada2ec4ec1adaec4ac14bbe368: Neither blocks nor log segments found. Creating new log.
I20260808 14:33:05.167508 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:33:05.167599 29585 log.cc:824] T 00000000000000000000000000000000 P 66dbdcada2ec4ec1adaec4ac14bbe368: Log is configured to *not* fsync() on all Append() calls
I20260808 14:33:05.169978 29585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66dbdcada2ec4ec1adaec4ac14bbe368: No bootstrap required, opened a new log
I20260808 14:33:05.174026 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:05.169741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51353 (local address 127.0.0.1:45233)
0808 14:33:05.169870 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:05.169876 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:05.170141 (+ 265us) server_negotiation.cc:407] Connection header received
0808 14:33:05.170209 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:05.170212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:05.170260 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:05.170341 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:05.170592 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:05.171349 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:05.172154 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:05.172611 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:05.172723 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:05.173567 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:05.173573 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:05.173577 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:05.173591 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:05.173621 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:05.173634 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:05.173717 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:05.173810 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 14:33:05.173868 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 14:33:05.174602 29585 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66dbdcada2ec4ec1adaec4ac14bbe368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66dbdcada2ec4ec1adaec4ac14bbe368" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45233 } }
I20260808 14:33:05.174916 29585 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66dbdcada2ec4ec1adaec4ac14bbe368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:33:05.175004 29585 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66dbdcada2ec4ec1adaec4ac14bbe368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66dbdcada2ec4ec1adaec4ac14bbe368, State: Initialized, Role: FOLLOWER
I20260808 14:33:05.175477 29585 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66dbdcada2ec4ec1adaec4ac14bbe368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66dbdcada2ec4ec1adaec4ac14bbe368" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45233 } }
I20260808 14:33:05.175611 29585 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66dbdcada2ec4ec1adaec4ac14bbe368 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:33:05.175660 29585 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66dbdcada2ec4ec1adaec4ac14bbe368 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:33:05.175735 29585 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66dbdcada2ec4ec1adaec4ac14bbe368 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:33:05.176704 29585 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66dbdcada2ec4ec1adaec4ac14bbe368 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66dbdcada2ec4ec1adaec4ac14bbe368" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45233 } }
I20260808 14:33:05.177031 29585 leader_election.cc:302] T 00000000000000000000000000000000 P 66dbdcada2ec4ec1adaec4ac14bbe368 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66dbdcada2ec4ec1adaec4ac14bbe368; no voters:
I20260808 14:33:05.177323 29585 leader_election.cc:288] T 00000000000000000000000000000000 P 66dbdcada2ec4ec1adaec4ac14bbe368 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:33:05.177457 29595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66dbdcada2ec4ec1adaec4ac14bbe368 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:33:05.177691 29595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66dbdcada2ec4ec1adaec4ac14bbe368 [term 1 LEADER]: Becoming Leader. State: Replica: 66dbdcada2ec4ec1adaec4ac14bbe368, State: Running, Role: LEADER
I20260808 14:33:05.178066 29595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66dbdcada2ec4ec1adaec4ac14bbe368 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66dbdcada2ec4ec1adaec4ac14bbe368" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45233 } }
I20260808 14:33:05.178663 29585 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66dbdcada2ec4ec1adaec4ac14bbe368 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:33:05.179807 29599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66dbdcada2ec4ec1adaec4ac14bbe368 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66dbdcada2ec4ec1adaec4ac14bbe368. Latest consensus state: current_term: 1 leader_uuid: "66dbdcada2ec4ec1adaec4ac14bbe368" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66dbdcada2ec4ec1adaec4ac14bbe368" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45233 } } }
I20260808 14:33:05.179963 29599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66dbdcada2ec4ec1adaec4ac14bbe368 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:05.180292 29598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66dbdcada2ec4ec1adaec4ac14bbe368 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66dbdcada2ec4ec1adaec4ac14bbe368" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66dbdcada2ec4ec1adaec4ac14bbe368" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45233 } } }
I20260808 14:33:05.180384 29598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66dbdcada2ec4ec1adaec4ac14bbe368 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:05.180478 29606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:33:05.183706 29606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:33:05.189781 29606 catalog_manager.cc:1347] Generated new cluster ID: 09adda3a924f46728c45d8af030647c7
I20260808 14:33:05.189862 29606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:33:05.202373 29606 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:33:05.203578 29606 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:33:05.209152 29606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66dbdcada2ec4ec1adaec4ac14bbe368: Generated new TSK 0
I20260808 14:33:05.210047 29606 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:33:05.213721 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 29510
2026-08-08T14:33:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1472 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:33:05 dist-test-slave-w206 krb5kdc[29619](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:33:05 dist-test-slave-w206 krb5kdc[29619](info): set up 2 sockets
Aug 08 14:33:05 dist-test-slave-w206 krb5kdc[29619](info): commencing operation
krb5kdc: starting...
W20260808 14:33:06.276515 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:33:06 dist-test-slave-w206 krb5kdc[29619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:33:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:33:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:33:06.406165 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39509
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:33:06.520632 29632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:33:06.520985 29632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:33:06.521075 29632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:33:06.525296 29632 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:33:06.525408 29632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:33:06.525442 29632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:33:06.525485 29632 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:33:06.525518 29632 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:33:06.531028 29632 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:33:06.532228 29632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:33:06.533320 29632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:33:06.542325 29638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:06.542325 29640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:06.542374 29637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:06.542745 29632 server_base.cc:1034] running on GCE node
I20260808 14:33:06.543352 29632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:33:06.544479 29632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:33:06.545667 29632 hybrid_clock.cc:648] HybridClock initialized: now 1786199586545634 us; error 24 us; skew 500 ppm
Aug 08 14:33:06 dist-test-slave-w206 krb5kdc[29619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199586, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:33:06.548504 29632 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:33:06.549624 29632 webserver.cc:460] Webserver started at http://127.0.0.1:41115/ using document root <none> and password file <none>
I20260808 14:33:06.550251 29632 fs_manager.cc:359] Metadata directory not provided
I20260808 14:33:06.550308 29632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:33:06.550477 29632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:33:06.552688 29632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "f39678f0ceed4710b0c65410528112c0"
format_stamp: "Formatted at 2026-08-08 14:33:06 on dist-test-slave-w206"
server_key: "7536d6dfc9c617afaacf9b0c686837a5"
server_key_iv: "30699956422ba718bfcc51666785b120"
server_key_version: "encryptionkey@0"
I20260808 14:33:06.553236 29632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "f39678f0ceed4710b0c65410528112c0"
format_stamp: "Formatted at 2026-08-08 14:33:06 on dist-test-slave-w206"
server_key: "7536d6dfc9c617afaacf9b0c686837a5"
server_key_iv: "30699956422ba718bfcc51666785b120"
server_key_version: "encryptionkey@0"
I20260808 14:33:06.557060 29632 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:33:06.559526 29647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:33:06.560768 29632 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:33:06.560880 29632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "f39678f0ceed4710b0c65410528112c0"
format_stamp: "Formatted at 2026-08-08 14:33:06 on dist-test-slave-w206"
server_key: "7536d6dfc9c617afaacf9b0c686837a5"
server_key_iv: "30699956422ba718bfcc51666785b120"
server_key_version: "encryptionkey@0"
I20260808 14:33:06.560973 29632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:33:06.565660 29632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:33:06.568709 29632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:33:06.568872 29632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:33:06.576052 29632 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39509
I20260808 14:33:06.576076 29702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39509 every 8 connection(s)
I20260808 14:33:06.577049 29632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:33:06.579900 29704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:33:06.583531 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 29632
I20260808 14:33:06.583680 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:33:06.583990 7669 external_mini_cluster.cc:1428] Setting key 5f1cfcf5e3ec3d8580e5b12642421d8f
I20260808 14:33:06.585958 29704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f39678f0ceed4710b0c65410528112c0: Bootstrap starting.
I20260808 14:33:06.588587 29704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f39678f0ceed4710b0c65410528112c0: Neither blocks nor log segments found. Creating new log.
I20260808 14:33:06.589376 29704 log.cc:824] T 00000000000000000000000000000000 P f39678f0ceed4710b0c65410528112c0: Log is configured to *not* fsync() on all Append() calls
I20260808 14:33:06.589385 29707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:06.585451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36503 (local address 127.0.0.1:39509)
0808 14:33:06.585898 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:06.585914 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:06.585950 (+ 36us) server_negotiation.cc:407] Connection header received
0808 14:33:06.586615 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:06.586632 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:06.586921 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:06.587263 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:06.588156 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:06.588160 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:06.588173 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:06.588183 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:06.588240 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:06.588260 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:06.588349 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:06.588499 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 14:33:06.588665 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":141,"threads_started":1}
I20260808 14:33:06.591567 29704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f39678f0ceed4710b0c65410528112c0: No bootstrap required, opened a new log
I20260808 14:33:06.592310 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:33:06.596614 29707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:06.594624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47617 (local address 127.0.0.1:39509)
0808 14:33:06.594759 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:06.594766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:06.595318 (+ 552us) server_negotiation.cc:407] Connection header received
0808 14:33:06.595441 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:06.595443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:06.595474 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:06.595537 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:06.596223 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:06.596226 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:06.596228 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:06.596238 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:06.596262 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:06.596271 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:06.596339 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:06.596457 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 14:33:06.596489 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 14:33:06.596413 29704 raft_consensus.cc:348] T 00000000000000000000000000000000 P f39678f0ceed4710b0c65410528112c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f39678f0ceed4710b0c65410528112c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39509 } }
I20260808 14:33:06.596784 29704 raft_consensus.cc:374] T 00000000000000000000000000000000 P f39678f0ceed4710b0c65410528112c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:33:06.596848 29704 raft_consensus.cc:729] T 00000000000000000000000000000000 P f39678f0ceed4710b0c65410528112c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f39678f0ceed4710b0c65410528112c0, State: Initialized, Role: FOLLOWER
I20260808 14:33:06.597321 29704 consensus_queue.cc:260] T 00000000000000000000000000000000 P f39678f0ceed4710b0c65410528112c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f39678f0ceed4710b0c65410528112c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39509 } }
I20260808 14:33:06.597471 29704 raft_consensus.cc:388] T 00000000000000000000000000000000 P f39678f0ceed4710b0c65410528112c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:33:06.597538 29704 raft_consensus.cc:482] T 00000000000000000000000000000000 P f39678f0ceed4710b0c65410528112c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:33:06.597640 29704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f39678f0ceed4710b0c65410528112c0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:33:06.598585 29704 raft_consensus.cc:504] T 00000000000000000000000000000000 P f39678f0ceed4710b0c65410528112c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f39678f0ceed4710b0c65410528112c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39509 } }
I20260808 14:33:06.598937 29704 leader_election.cc:302] T 00000000000000000000000000000000 P f39678f0ceed4710b0c65410528112c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f39678f0ceed4710b0c65410528112c0; no voters:
I20260808 14:33:06.599323 29704 leader_election.cc:288] T 00000000000000000000000000000000 P f39678f0ceed4710b0c65410528112c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:33:06.599447 29715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f39678f0ceed4710b0c65410528112c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:33:06.599700 29715 raft_consensus.cc:686] T 00000000000000000000000000000000 P f39678f0ceed4710b0c65410528112c0 [term 1 LEADER]: Becoming Leader. State: Replica: f39678f0ceed4710b0c65410528112c0, State: Running, Role: LEADER
I20260808 14:33:06.600066 29715 consensus_queue.cc:237] T 00000000000000000000000000000000 P f39678f0ceed4710b0c65410528112c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f39678f0ceed4710b0c65410528112c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39509 } }
I20260808 14:33:06.600551 29704 sys_catalog.cc:564] T 00000000000000000000000000000000 P f39678f0ceed4710b0c65410528112c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:33:06.601922 29717 sys_catalog.cc:455] T 00000000000000000000000000000000 P f39678f0ceed4710b0c65410528112c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f39678f0ceed4710b0c65410528112c0. Latest consensus state: current_term: 1 leader_uuid: "f39678f0ceed4710b0c65410528112c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f39678f0ceed4710b0c65410528112c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39509 } } }
I20260808 14:33:06.602053 29717 sys_catalog.cc:458] T 00000000000000000000000000000000 P f39678f0ceed4710b0c65410528112c0 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:06.602474 29726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:33:06.602424 29716 sys_catalog.cc:455] T 00000000000000000000000000000000 P f39678f0ceed4710b0c65410528112c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f39678f0ceed4710b0c65410528112c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f39678f0ceed4710b0c65410528112c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39509 } } }
I20260808 14:33:06.602588 29716 sys_catalog.cc:458] T 00000000000000000000000000000000 P f39678f0ceed4710b0c65410528112c0 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:06.605607 29726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:33:06.611177 29726 catalog_manager.cc:1347] Generated new cluster ID: 530264c0d206471db4132c650f071185
I20260808 14:33:06.611243 29726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:33:06.617347 29726 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:33:06.617681 29726 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:33:06.633671 29726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f39678f0ceed4710b0c65410528112c0: Generated new TSK 0
I20260808 14:33:06.634567 29726 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:33:06.641198 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 29632
2026-08-08T14:33:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:33:06 dist-test-slave-w206 krb5kdc[29736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:33:06 dist-test-slave-w206 krb5kdc[29736](info): set up 2 sockets
Aug 08 14:33:06 dist-test-slave-w206 krb5kdc[29736](info): commencing operation
krb5kdc: starting...
W20260808 14:33:07.739800 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:33:07 dist-test-slave-w206 krb5kdc[29736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:33:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:33:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:33:07.873773 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46575
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:33:07.989271 29751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:33:07.989605 29751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:33:07.989691 29751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:33:07.993770 29751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:33:07.993885 29751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:33:07.993927 29751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:33:07.993965 29751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:33:07.993999 29751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:33:07.999467 29751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:33:08.000687 29751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:33:08.001797 29751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:33:08.011075 29756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:08.011068 29757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:08.011068 29759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:08.011188 29751 server_base.cc:1034] running on GCE node
I20260808 14:33:08.011965 29751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:33:08.013118 29751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:33:08.014309 29751 hybrid_clock.cc:648] HybridClock initialized: now 1786199588014281 us; error 44 us; skew 500 ppm
Aug 08 14:33:08 dist-test-slave-w206 krb5kdc[29736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199588, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:33:08.017225 29751 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:33:08.018095 29751 webserver.cc:460] Webserver started at http://127.0.0.1:39571/ using document root <none> and password file <none>
I20260808 14:33:08.018611 29751 fs_manager.cc:359] Metadata directory not provided
I20260808 14:33:08.018668 29751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:33:08.018839 29751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:33:08.021049 29751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "c0962e30ab5744bb84491889c6399a05"
format_stamp: "Formatted at 2026-08-08 14:33:08 on dist-test-slave-w206"
server_key: "e8f02b91b850031dc60d64c76e0fc87a"
server_key_iv: "ab24d798186e51810ab1772a713222e4"
server_key_version: "encryptionkey@0"
I20260808 14:33:08.021577 29751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "c0962e30ab5744bb84491889c6399a05"
format_stamp: "Formatted at 2026-08-08 14:33:08 on dist-test-slave-w206"
server_key: "e8f02b91b850031dc60d64c76e0fc87a"
server_key_iv: "ab24d798186e51810ab1772a713222e4"
server_key_version: "encryptionkey@0"
I20260808 14:33:08.025359 29751 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260808 14:33:08.027794 29766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:33:08.029136 29751 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260808 14:33:08.029244 29751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "c0962e30ab5744bb84491889c6399a05"
format_stamp: "Formatted at 2026-08-08 14:33:08 on dist-test-slave-w206"
server_key: "e8f02b91b850031dc60d64c76e0fc87a"
server_key_iv: "ab24d798186e51810ab1772a713222e4"
server_key_version: "encryptionkey@0"
I20260808 14:33:08.029336 29751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:33:08.052139 29751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:33:08.055740 29751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:33:08.055943 29751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:33:08.063627 29751 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46575
I20260808 14:33:08.063638 29819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46575 every 8 connection(s)
I20260808 14:33:08.064718 29751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:33:08.067647 29820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:33:08.071573 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 29751
I20260808 14:33:08.071712 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:33:08.072010 7669 external_mini_cluster.cc:1428] Setting key c2da01bb927a2937ec274eed4425e250
I20260808 14:33:08.073980 29820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0962e30ab5744bb84491889c6399a05: Bootstrap starting.
I20260808 14:33:08.076613 29820 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0962e30ab5744bb84491889c6399a05: Neither blocks nor log segments found. Creating new log.
I20260808 14:33:08.077437 29820 log.cc:824] T 00000000000000000000000000000000 P c0962e30ab5744bb84491889c6399a05: Log is configured to *not* fsync() on all Append() calls
I20260808 14:33:08.079563 29820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0962e30ab5744bb84491889c6399a05: No bootstrap required, opened a new log
Aug 08 14:33:08 dist-test-slave-w206 krb5kdc[29736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 14:33:08.083142 29820 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0962e30ab5744bb84491889c6399a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0962e30ab5744bb84491889c6399a05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46575 } }
I20260808 14:33:08.083458 29820 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0962e30ab5744bb84491889c6399a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:33:08.083539 29820 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0962e30ab5744bb84491889c6399a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0962e30ab5744bb84491889c6399a05, State: Initialized, Role: FOLLOWER
I20260808 14:33:08.084055 29820 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0962e30ab5744bb84491889c6399a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0962e30ab5744bb84491889c6399a05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46575 } }
I20260808 14:33:08.084185 29820 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0962e30ab5744bb84491889c6399a05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:33:08.084270 29820 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0962e30ab5744bb84491889c6399a05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:33:08.084399 29820 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0962e30ab5744bb84491889c6399a05 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:33:08.085345 29820 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0962e30ab5744bb84491889c6399a05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0962e30ab5744bb84491889c6399a05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46575 } }
I20260808 14:33:08.085716 29820 leader_election.cc:302] T 00000000000000000000000000000000 P c0962e30ab5744bb84491889c6399a05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0962e30ab5744bb84491889c6399a05; no voters:
I20260808 14:33:08.086076 29820 leader_election.cc:288] T 00000000000000000000000000000000 P c0962e30ab5744bb84491889c6399a05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:33:08.086448 29826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0962e30ab5744bb84491889c6399a05 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:33:08.086908 29826 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0962e30ab5744bb84491889c6399a05 [term 1 LEADER]: Becoming Leader. State: Replica: c0962e30ab5744bb84491889c6399a05, State: Running, Role: LEADER
I20260808 14:33:08.087245 29826 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0962e30ab5744bb84491889c6399a05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0962e30ab5744bb84491889c6399a05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46575 } }
I20260808 14:33:08.087507 29820 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0962e30ab5744bb84491889c6399a05 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:33:08.087839 29823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:08.073511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:52829 (local address 127.0.0.1:46575)
0808 14:33:08.073943 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:08.073960 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:08.073997 (+ 37us) server_negotiation.cc:407] Connection header received
0808 14:33:08.074683 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:08.074700 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:08.075120 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:08.075413 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:08.076389 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:08.077456 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:08.078368 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:08.078844 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:08.079045 (+ 201us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:08.081911 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:08.081915 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:08.081930 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:08.081959 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:08.084362 (+ 2403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:08.084908 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:08.084915 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:08.084921 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:08.085047 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:08.085422 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:08.085426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:08.085428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:08.085967 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:08.086201 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:08.086576 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 14:33:08.086842 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":140,"threads_started":1}
I20260808 14:33:08.088517 29826 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0962e30ab5744bb84491889c6399a05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0962e30ab5744bb84491889c6399a05. Latest consensus state: current_term: 1 leader_uuid: "c0962e30ab5744bb84491889c6399a05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0962e30ab5744bb84491889c6399a05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46575 } } }
I20260808 14:33:08.088711 29826 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0962e30ab5744bb84491889c6399a05 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:08.088922 29825 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0962e30ab5744bb84491889c6399a05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0962e30ab5744bb84491889c6399a05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0962e30ab5744bb84491889c6399a05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46575 } } }
I20260808 14:33:08.089023 29825 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0962e30ab5744bb84491889c6399a05 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:08.089177 29834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:33:08.091144 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:33:08.092445 29834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:33:08.098284 29834 catalog_manager.cc:1347] Generated new cluster ID: 7ce7979214944763b9ff40b68b1b622b
I20260808 14:33:08.099066 29834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:33:08.103202 29823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:08.093427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:45773 (local address 127.0.0.1:46575)
0808 14:33:08.093563 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:08.093570 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:08.094042 (+ 472us) server_negotiation.cc:407] Connection header received
0808 14:33:08.094230 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:08.094234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:08.094284 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:08.094340 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:08.095352 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:08.096097 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:08.097300 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:08.097631 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:08.097738 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:08.099548 (+ 1810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:08.099552 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:08.099555 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:08.099580 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:08.101302 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:08.101843 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:08.101847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:08.101848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:08.101938 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:08.102298 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:08.102301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:08.102302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:08.102496 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:08.102618 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:08.102932 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 14:33:08.103050 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 14:33:08.106921 29834 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:33:08.107901 29834 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:33:08.111833 29834 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0962e30ab5744bb84491889c6399a05: Generated new TSK 0
I20260808 14:33:08.112516 29834 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:33:08.119215 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 29751
W20260808 14:33:08.126319 29734 connection.cc:535] client connection to 127.0.0.1:46575 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46575): Connection reset by peer (error 104)
2026-08-08T14:33:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1478 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:33:08 dist-test-slave-w206 krb5kdc[29852](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:33:08 dist-test-slave-w206 krb5kdc[29852](info): set up 2 sockets
Aug 08 14:33:08 dist-test-slave-w206 krb5kdc[29852](info): commencing operation
krb5kdc: starting...
W20260808 14:33:09.220361 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:33:09 dist-test-slave-w206 krb5kdc[29852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:33:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:33:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:33:09.353680 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38467
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:33:09.467589 29865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:33:09.467855 29865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:33:09.467898 29865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:33:09.471882 29865 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:33:09.471946 29865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:33:09.471966 29865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:33:09.471982 29865 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:33:09.471997 29865 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:33:09.477308 29865 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:33:09.478351 29865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:33:09.479403 29865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:33:09.488058 29873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:09.488061 29870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:09.488096 29865 server_base.cc:1034] running on GCE node
W20260808 14:33:09.488052 29871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:09.488853 29865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:33:09.489913 29865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:33:09.491052 29865 hybrid_clock.cc:648] HybridClock initialized: now 1786199589491025 us; error 40 us; skew 500 ppm
Aug 08 14:33:09 dist-test-slave-w206 krb5kdc[29852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199589, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:33:09.493875 29865 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:33:09.494726 29865 webserver.cc:460] Webserver started at http://127.0.0.1:43291/ using document root <none> and password file <none>
I20260808 14:33:09.495386 29865 fs_manager.cc:359] Metadata directory not provided
I20260808 14:33:09.495482 29865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:33:09.495710 29865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:33:09.497752 29865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "eaa3d51d3801435aaf1a0f1aef42cf6e"
format_stamp: "Formatted at 2026-08-08 14:33:09 on dist-test-slave-w206"
server_key: "62be938f8036a0250264e1cc63daa745"
server_key_iv: "a4559656daf7b4053c440468e6c41d9f"
server_key_version: "encryptionkey@0"
I20260808 14:33:09.498311 29865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "eaa3d51d3801435aaf1a0f1aef42cf6e"
format_stamp: "Formatted at 2026-08-08 14:33:09 on dist-test-slave-w206"
server_key: "62be938f8036a0250264e1cc63daa745"
server_key_iv: "a4559656daf7b4053c440468e6c41d9f"
server_key_version: "encryptionkey@0"
I20260808 14:33:09.502076 29865 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 14:33:09.504519 29882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:33:09.505722 29865 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:33:09.505870 29865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "eaa3d51d3801435aaf1a0f1aef42cf6e"
format_stamp: "Formatted at 2026-08-08 14:33:09 on dist-test-slave-w206"
server_key: "62be938f8036a0250264e1cc63daa745"
server_key_iv: "a4559656daf7b4053c440468e6c41d9f"
server_key_version: "encryptionkey@0"
I20260808 14:33:09.505991 29865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:33:09.516738 29865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:33:09.519948 29865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:33:09.520184 29865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:33:09.527630 29865 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38467
I20260808 14:33:09.527676 29939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38467 every 8 connection(s)
I20260808 14:33:09.528641 29865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:33:09.531086 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 29865
I20260808 14:33:09.531234 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:33:09.531534 7669 external_mini_cluster.cc:1428] Setting key 4894b9a5aa1c8a0f284ecbe649f08d6f
I20260808 14:33:09.531818 29940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:33:09.539937 29940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaa3d51d3801435aaf1a0f1aef42cf6e: Bootstrap starting.
I20260808 14:33:09.542533 29943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:09.533043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:52727 (local address 127.0.0.1:38467)
0808 14:33:09.533507 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:09.533529 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:09.533576 (+ 47us) server_negotiation.cc:407] Connection header received
0808 14:33:09.534417 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:09.534443 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:09.534950 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:09.535380 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:09.535847 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:09.537526 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:09.538436 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:09.539263 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:09.539435 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:09.540379 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:09.540386 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:09.540403 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:09.540964 (+ 561us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:09.541036 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:09.541069 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:09.541221 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:09.541357 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 14:33:09.541511 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":125,"threads_started":1}
I20260808 14:33:09.542955 29940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eaa3d51d3801435aaf1a0f1aef42cf6e: Neither blocks nor log segments found. Creating new log.
I20260808 14:33:09.543920 29940 log.cc:824] T 00000000000000000000000000000000 P eaa3d51d3801435aaf1a0f1aef42cf6e: Log is configured to *not* fsync() on all Append() calls
I20260808 14:33:09.545858 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:33:09.546072 29940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaa3d51d3801435aaf1a0f1aef42cf6e: No bootstrap required, opened a new log
I20260808 14:33:09.550407 29940 raft_consensus.cc:348] T 00000000000000000000000000000000 P eaa3d51d3801435aaf1a0f1aef42cf6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa3d51d3801435aaf1a0f1aef42cf6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38467 } }
I20260808 14:33:09.550683 29940 raft_consensus.cc:374] T 00000000000000000000000000000000 P eaa3d51d3801435aaf1a0f1aef42cf6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:33:09.550735 29940 raft_consensus.cc:729] T 00000000000000000000000000000000 P eaa3d51d3801435aaf1a0f1aef42cf6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaa3d51d3801435aaf1a0f1aef42cf6e, State: Initialized, Role: FOLLOWER
I20260808 14:33:09.551244 29940 consensus_queue.cc:260] T 00000000000000000000000000000000 P eaa3d51d3801435aaf1a0f1aef42cf6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa3d51d3801435aaf1a0f1aef42cf6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38467 } }
I20260808 14:33:09.551410 29940 raft_consensus.cc:388] T 00000000000000000000000000000000 P eaa3d51d3801435aaf1a0f1aef42cf6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:33:09.551486 29940 raft_consensus.cc:482] T 00000000000000000000000000000000 P eaa3d51d3801435aaf1a0f1aef42cf6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:33:09.551592 29940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eaa3d51d3801435aaf1a0f1aef42cf6e [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:33:09.552779 29943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:09.548167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:45143 (local address 127.0.0.1:38467)
0808 14:33:09.548301 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:09.548305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:09.548462 (+ 157us) server_negotiation.cc:407] Connection header received
0808 14:33:09.548583 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:09.548586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:09.548634 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:09.548719 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:09.548962 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:09.549737 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:09.550602 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:09.550938 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:09.551078 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:09.552165 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:09.552171 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:09.552178 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:09.552192 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:09.552243 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:09.552258 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:09.552353 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:09.552562 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 14:33:09.552635 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 14:33:09.553006 29940 raft_consensus.cc:504] T 00000000000000000000000000000000 P eaa3d51d3801435aaf1a0f1aef42cf6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa3d51d3801435aaf1a0f1aef42cf6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38467 } }
I20260808 14:33:09.553381 29940 leader_election.cc:302] T 00000000000000000000000000000000 P eaa3d51d3801435aaf1a0f1aef42cf6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eaa3d51d3801435aaf1a0f1aef42cf6e; no voters:
I20260808 14:33:09.553723 29940 leader_election.cc:288] T 00000000000000000000000000000000 P eaa3d51d3801435aaf1a0f1aef42cf6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:33:09.553825 29950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eaa3d51d3801435aaf1a0f1aef42cf6e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:33:09.554018 29950 raft_consensus.cc:686] T 00000000000000000000000000000000 P eaa3d51d3801435aaf1a0f1aef42cf6e [term 1 LEADER]: Becoming Leader. State: Replica: eaa3d51d3801435aaf1a0f1aef42cf6e, State: Running, Role: LEADER
I20260808 14:33:09.554329 29950 consensus_queue.cc:237] T 00000000000000000000000000000000 P eaa3d51d3801435aaf1a0f1aef42cf6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa3d51d3801435aaf1a0f1aef42cf6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38467 } }
I20260808 14:33:09.555111 29940 sys_catalog.cc:564] T 00000000000000000000000000000000 P eaa3d51d3801435aaf1a0f1aef42cf6e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:33:09.556219 29951 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaa3d51d3801435aaf1a0f1aef42cf6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eaa3d51d3801435aaf1a0f1aef42cf6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa3d51d3801435aaf1a0f1aef42cf6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38467 } } }
I20260808 14:33:09.556382 29951 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaa3d51d3801435aaf1a0f1aef42cf6e [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:09.556690 29952 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaa3d51d3801435aaf1a0f1aef42cf6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader eaa3d51d3801435aaf1a0f1aef42cf6e. Latest consensus state: current_term: 1 leader_uuid: "eaa3d51d3801435aaf1a0f1aef42cf6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa3d51d3801435aaf1a0f1aef42cf6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38467 } } }
I20260808 14:33:09.556778 29952 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaa3d51d3801435aaf1a0f1aef42cf6e [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:09.556814 29960 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:33:09.559857 29960 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:33:09.565411 29960 catalog_manager.cc:1347] Generated new cluster ID: 687132d9b7564b72bcf1118b03622060
I20260808 14:33:09.565482 29960 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:33:09.582053 29960 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:33:09.583458 29960 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:33:09.592998 29960 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eaa3d51d3801435aaf1a0f1aef42cf6e: Generated new TSK 0
I20260808 14:33:09.593888 29960 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:33:09.600134 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 29865
W20260808 14:33:09.608222 29850 connection.cc:535] client connection to 127.0.0.1:38467 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38467): Connection reset by peer (error 104)
2026-08-08T14:33:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1481 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:33:09 dist-test-slave-w206 krb5kdc[29973](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:33:09 dist-test-slave-w206 krb5kdc[29973](info): set up 2 sockets
Aug 08 14:33:09 dist-test-slave-w206 krb5kdc[29973](info): commencing operation
krb5kdc: starting...
W20260808 14:33:10.663741 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:33:10 dist-test-slave-w206 krb5kdc[29973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:33:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:33:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:33:10.794384 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38295
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:33:10.906668 29987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:33:10.906984 29987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:33:10.907081 29987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:33:10.911054 29987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:33:10.911147 29987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:33:10.911187 29987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:33:10.911223 29987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:33:10.911260 29987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:33:10.916448 29987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:33:10.917541 29987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:33:10.918586 29987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:33:10.927158 29995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:10.927146 29992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:10.927173 29993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:10.927248 29987 server_base.cc:1034] running on GCE node
I20260808 14:33:10.928090 29987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:33:10.929219 29987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:33:10.930416 29987 hybrid_clock.cc:648] HybridClock initialized: now 1786199590930398 us; error 31 us; skew 500 ppm
Aug 08 14:33:10 dist-test-slave-w206 krb5kdc[29973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199590, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:33:10.938258 29987 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:33:10.939195 29987 webserver.cc:460] Webserver started at http://127.0.0.1:36463/ using document root <none> and password file <none>
I20260808 14:33:10.939718 29987 fs_manager.cc:359] Metadata directory not provided
I20260808 14:33:10.939775 29987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:33:10.939973 29987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:33:10.941975 29987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "2ebaf4a7975049c2a11b195604eb7a80"
format_stamp: "Formatted at 2026-08-08 14:33:10 on dist-test-slave-w206"
server_key: "08e0802e16258b357f505676e770b998"
server_key_iv: "525b2e33b7a42e7245428648f5532039"
server_key_version: "encryptionkey@0"
I20260808 14:33:10.942485 29987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "2ebaf4a7975049c2a11b195604eb7a80"
format_stamp: "Formatted at 2026-08-08 14:33:10 on dist-test-slave-w206"
server_key: "08e0802e16258b357f505676e770b998"
server_key_iv: "525b2e33b7a42e7245428648f5532039"
server_key_version: "encryptionkey@0"
I20260808 14:33:10.946208 29987 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:33:10.948679 30002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:33:10.949878 29987 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:33:10.950009 29987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "2ebaf4a7975049c2a11b195604eb7a80"
format_stamp: "Formatted at 2026-08-08 14:33:10 on dist-test-slave-w206"
server_key: "08e0802e16258b357f505676e770b998"
server_key_iv: "525b2e33b7a42e7245428648f5532039"
server_key_version: "encryptionkey@0"
I20260808 14:33:10.950124 29987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:33:10.956313 29987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:33:10.959307 29987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:33:10.959502 29987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:33:10.966797 29987 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38295
I20260808 14:33:10.966888 30057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38295 every 8 connection(s)
I20260808 14:33:10.967823 29987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:33:10.970583 30058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:33:10.971525 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 29987
I20260808 14:33:10.971668 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:33:10.971997 7669 external_mini_cluster.cc:1428] Setting key 22caaa043c0fa11f557a7c5ccd5a93b2
I20260808 14:33:10.977501 30061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:10.973272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:37959 (local address 127.0.0.1:38295)
0808 14:33:10.973700 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:10.973719 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:10.973763 (+ 44us) server_negotiation.cc:407] Connection header received
0808 14:33:10.974577 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:10.974601 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:10.974945 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:10.975333 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:10.976102 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:10.976108 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:10.976123 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:10.976137 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:10.976203 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:10.976234 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:10.976384 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:10.976453 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 14:33:10.976609 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":108,"threads_started":1}
I20260808 14:33:10.977516 30058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ebaf4a7975049c2a11b195604eb7a80: Bootstrap starting.
I20260808 14:33:10.980237 30058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ebaf4a7975049c2a11b195604eb7a80: Neither blocks nor log segments found. Creating new log.
I20260808 14:33:10.980687 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:33:10.981084 30058 log.cc:824] T 00000000000000000000000000000000 P 2ebaf4a7975049c2a11b195604eb7a80: Log is configured to *not* fsync() on all Append() calls
I20260808 14:33:10.983223 30058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ebaf4a7975049c2a11b195604eb7a80: No bootstrap required, opened a new log
I20260808 14:33:10.984812 30061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:10.983061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:54449 (local address 127.0.0.1:38295)
0808 14:33:10.983170 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:10.983174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:10.983431 (+ 257us) server_negotiation.cc:407] Connection header received
0808 14:33:10.983545 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:10.983547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:10.983578 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:10.983645 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:10.984377 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:10.984380 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:10.984382 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:10.984392 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:10.984427 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:10.984437 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:10.984508 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:10.984618 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 14:33:10.984654 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 14:33:10.987504 30058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ebaf4a7975049c2a11b195604eb7a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebaf4a7975049c2a11b195604eb7a80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38295 } }
I20260808 14:33:10.987728 30058 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ebaf4a7975049c2a11b195604eb7a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:33:10.987803 30058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ebaf4a7975049c2a11b195604eb7a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ebaf4a7975049c2a11b195604eb7a80, State: Initialized, Role: FOLLOWER
I20260808 14:33:10.988263 30058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ebaf4a7975049c2a11b195604eb7a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebaf4a7975049c2a11b195604eb7a80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38295 } }
I20260808 14:33:10.988407 30058 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ebaf4a7975049c2a11b195604eb7a80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:33:10.988474 30058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ebaf4a7975049c2a11b195604eb7a80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:33:10.988593 30058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ebaf4a7975049c2a11b195604eb7a80 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:33:10.989547 30058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ebaf4a7975049c2a11b195604eb7a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebaf4a7975049c2a11b195604eb7a80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38295 } }
I20260808 14:33:10.989900 30058 leader_election.cc:302] T 00000000000000000000000000000000 P 2ebaf4a7975049c2a11b195604eb7a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ebaf4a7975049c2a11b195604eb7a80; no voters:
I20260808 14:33:10.990267 30058 leader_election.cc:288] T 00000000000000000000000000000000 P 2ebaf4a7975049c2a11b195604eb7a80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:33:10.990324 30069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ebaf4a7975049c2a11b195604eb7a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:33:10.990566 30069 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ebaf4a7975049c2a11b195604eb7a80 [term 1 LEADER]: Becoming Leader. State: Replica: 2ebaf4a7975049c2a11b195604eb7a80, State: Running, Role: LEADER
I20260808 14:33:10.990917 30069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ebaf4a7975049c2a11b195604eb7a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebaf4a7975049c2a11b195604eb7a80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38295 } }
I20260808 14:33:10.991676 30058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ebaf4a7975049c2a11b195604eb7a80 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:33:10.992786 30071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ebaf4a7975049c2a11b195604eb7a80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ebaf4a7975049c2a11b195604eb7a80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebaf4a7975049c2a11b195604eb7a80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38295 } } }
I20260808 14:33:10.992839 30070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ebaf4a7975049c2a11b195604eb7a80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ebaf4a7975049c2a11b195604eb7a80. Latest consensus state: current_term: 1 leader_uuid: "2ebaf4a7975049c2a11b195604eb7a80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ebaf4a7975049c2a11b195604eb7a80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38295 } } }
I20260808 14:33:10.992933 30071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ebaf4a7975049c2a11b195604eb7a80 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:10.992944 30070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ebaf4a7975049c2a11b195604eb7a80 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:10.993381 30078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:33:10.996804 30078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:33:11.002569 30078 catalog_manager.cc:1347] Generated new cluster ID: fb2b82961b294e6ba83cb75b04bdaa0e
I20260808 14:33:11.002655 30078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:33:11.025590 30078 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:33:11.025977 30078 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:33:11.031399 30078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ebaf4a7975049c2a11b195604eb7a80: Generated new TSK 0
I20260808 14:33:11.032202 30078 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:33:11.041864 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 29987
2026-08-08T14:33:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1441 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:33:11 dist-test-slave-w206 krb5kdc[30090](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:33:11 dist-test-slave-w206 krb5kdc[30090](info): set up 2 sockets
Aug 08 14:33:11 dist-test-slave-w206 krb5kdc[30090](info): commencing operation
krb5kdc: starting...
W20260808 14:33:12.151787 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:33:12 dist-test-slave-w206 krb5kdc[30090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:33:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:33:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:33:12.290581 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33313
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:33:12.405265 30105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:33:12.405598 30105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:33:12.405700 30105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:33:12.410126 30105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:33:12.410247 30105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:33:12.410290 30105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:33:12.410328 30105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:33:12.410373 30105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:33:12.416100 30105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33313
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:33:12.417290 30105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:33:12.418352 30105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:33:12.427224 30105 server_base.cc:1034] running on GCE node
W20260808 14:33:12.427122 30110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:12.427138 30111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:12.427122 30113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:12.428005 30105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:33:12.429159 30105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:33:12.430323 30105 hybrid_clock.cc:648] HybridClock initialized: now 1786199592430292 us; error 43 us; skew 500 ppm
Aug 08 14:33:12 dist-test-slave-w206 krb5kdc[30090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199592, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:33:12.433293 30105 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:33:12.434172 30105 webserver.cc:460] Webserver started at http://127.0.0.1:42557/ using document root <none> and password file <none>
I20260808 14:33:12.434728 30105 fs_manager.cc:359] Metadata directory not provided
I20260808 14:33:12.434791 30105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:33:12.435094 30105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:33:12.437196 30105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "1c1abeb0f6e34291b09d09339aa1e5cb"
format_stamp: "Formatted at 2026-08-08 14:33:12 on dist-test-slave-w206"
server_key: "1f95be6d38fd72cd1ee061901249e6b7"
server_key_iv: "2489591418acd55bb9cc9f017f269f72"
server_key_version: "encryptionkey@0"
I20260808 14:33:12.437786 30105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "1c1abeb0f6e34291b09d09339aa1e5cb"
format_stamp: "Formatted at 2026-08-08 14:33:12 on dist-test-slave-w206"
server_key: "1f95be6d38fd72cd1ee061901249e6b7"
server_key_iv: "2489591418acd55bb9cc9f017f269f72"
server_key_version: "encryptionkey@0"
I20260808 14:33:12.441638 30105 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:33:12.444162 30120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:33:12.445411 30105 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:33:12.445561 30105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "1c1abeb0f6e34291b09d09339aa1e5cb"
format_stamp: "Formatted at 2026-08-08 14:33:12 on dist-test-slave-w206"
server_key: "1f95be6d38fd72cd1ee061901249e6b7"
server_key_iv: "2489591418acd55bb9cc9f017f269f72"
server_key_version: "encryptionkey@0"
I20260808 14:33:12.445679 30105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:33:12.471702 30105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:33:12.475247 30105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:33:12.475456 30105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:33:12.483081 30105 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33313
I20260808 14:33:12.483108 30174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33313 every 8 connection(s)
I20260808 14:33:12.484045 30105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:33:12.486774 30175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:33:12.488469 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 30105
I20260808 14:33:12.488633 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:33:12.488960 7669 external_mini_cluster.cc:1428] Setting key 35bf944712d758e734ca4bba3863cc9d
I20260808 14:33:12.493669 30175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb: Bootstrap starting.
I20260808 14:33:12.496269 30175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb: Neither blocks nor log segments found. Creating new log.
I20260808 14:33:12.497191 30175 log.cc:824] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:33:12 dist-test-slave-w206 krb5kdc[30090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 14:33:12.499399 30175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb: No bootstrap required, opened a new log
I20260808 14:33:12.504338 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:12.490649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:56543 (local address 127.0.0.1:33313)
0808 14:33:12.491099 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:12.491115 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:12.491151 (+ 36us) server_negotiation.cc:407] Connection header received
0808 14:33:12.491796 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:12.491812 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:12.492159 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:12.492435 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:12.493376 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:12.494439 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:12.495647 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:12.496105 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:12.496218 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:12.499038 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:12.499046 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:12.499062 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:12.499093 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:12.501085 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:12.501619 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:12.501625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:12.501631 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:12.501788 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:12.502117 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:12.502133 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:12.502135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:12.502663 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:12.502878 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:12.503129 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 14:33:12.503373 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":144,"threads_started":1}
I20260808 14:33:12.504328 30175 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c1abeb0f6e34291b09d09339aa1e5cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33313 } }
I20260808 14:33:12.504616 30175 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:33:12.504674 30175 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c1abeb0f6e34291b09d09339aa1e5cb, State: Initialized, Role: FOLLOWER
I20260808 14:33:12.505149 30175 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c1abeb0f6e34291b09d09339aa1e5cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33313 } }
I20260808 14:33:12.505306 30175 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:33:12.505421 30175 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:33:12.505524 30175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:33:12.506618 30175 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c1abeb0f6e34291b09d09339aa1e5cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33313 } }
I20260808 14:33:12.507057 30175 leader_election.cc:302] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c1abeb0f6e34291b09d09339aa1e5cb; no voters:
I20260808 14:33:12.507404 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:33:12.507439 30175 leader_election.cc:288] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:33:12.507578 30180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:33:12.507815 30180 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb [term 1 LEADER]: Becoming Leader. State: Replica: 1c1abeb0f6e34291b09d09339aa1e5cb, State: Running, Role: LEADER
I20260808 14:33:12.508159 30180 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c1abeb0f6e34291b09d09339aa1e5cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33313 } }
I20260808 14:33:12.508759 30175 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:33:12.509886 30182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c1abeb0f6e34291b09d09339aa1e5cb. Latest consensus state: current_term: 1 leader_uuid: "1c1abeb0f6e34291b09d09339aa1e5cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c1abeb0f6e34291b09d09339aa1e5cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33313 } } }
I20260808 14:33:12.510006 30182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:12.510339 30181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c1abeb0f6e34291b09d09339aa1e5cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c1abeb0f6e34291b09d09339aa1e5cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33313 } } }
I20260808 14:33:12.510449 30181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb [sys.catalog]: This master's current role is: LEADER
W20260808 14:33:12.514031 30200 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:33:12.514117 30200 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:33:12.514173 30194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:33:12.514982 30194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:33:12.520711 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:12.509694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:41431 (local address 127.0.0.1:33313)
0808 14:33:12.509792 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:12.509797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:12.511061 (+ 1264us) server_negotiation.cc:407] Connection header received
0808 14:33:12.511521 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:12.511524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:12.511572 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:12.511654 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:12.512592 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:12.513346 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:12.514466 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:12.514802 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:12.514925 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:12.516685 (+ 1760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:12.516690 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:12.516693 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:12.516720 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:12.518558 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:12.519143 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:12.519147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:12.519149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:12.519244 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:12.519667 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:12.519670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:12.519672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:12.519929 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:12.520080 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:12.520435 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 14:33:12.520545 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 14:33:12.521994 30194 catalog_manager.cc:1347] Generated new cluster ID: e262f31e462e47eba96a7b88868a3f52
I20260808 14:33:12.522058 30194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:33:12.541832 30194 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:33:12.542816 30194 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:33:12.548944 30194 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c1abeb0f6e34291b09d09339aa1e5cb: Generated new TSK 0
I20260808 14:33:12.549697 30194 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:33:12.559733 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 30105
W20260808 14:33:12.567190 30088 connection.cc:535] client connection to 127.0.0.1:33313 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33313): Connection reset by peer (error 104)
2026-08-08T14:33:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1517 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:33:12 dist-test-slave-w206 krb5kdc[30206](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:33:12 dist-test-slave-w206 krb5kdc[30206](info): set up 2 sockets
Aug 08 14:33:12 dist-test-slave-w206 krb5kdc[30206](info): commencing operation
krb5kdc: starting...
W20260808 14:33:13.647423 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:33:13 dist-test-slave-w206 krb5kdc[30206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:33:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:33:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:33:13.783394 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40719
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:33:13.915421 30220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:33:13.915731 30220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:33:13.915786 30220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:33:13.919968 30220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:33:13.920044 30220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:33:13.920065 30220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:33:13.920083 30220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:33:13.920099 30220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:33:13.925760 30220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40719
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:33:13.926887 30220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:33:13.927973 30220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:33:13.937126 30225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:13.937090 30228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:13.937104 30226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:13.937177 30220 server_base.cc:1034] running on GCE node
I20260808 14:33:13.937939 30220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:33:13.939172 30220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:33:13.940366 30220 hybrid_clock.cc:648] HybridClock initialized: now 1786199593940356 us; error 27 us; skew 500 ppm
Aug 08 14:33:13 dist-test-slave-w206 krb5kdc[30206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199593, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:33:13.943441 30220 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:33:13.944278 30220 webserver.cc:460] Webserver started at http://127.0.0.1:45065/ using document root <none> and password file <none>
I20260808 14:33:13.944808 30220 fs_manager.cc:359] Metadata directory not provided
I20260808 14:33:13.944864 30220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:33:13.945034 30220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:33:13.947122 30220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "48bee00512c34031ab06ac68eeca64a7"
format_stamp: "Formatted at 2026-08-08 14:33:13 on dist-test-slave-w206"
server_key: "2e7e917d53212aa7578eeced5c6057d2"
server_key_iv: "97f645ef8408d47e5ec9b1ab1d7631e3"
server_key_version: "encryptionkey@0"
I20260808 14:33:13.947660 30220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "48bee00512c34031ab06ac68eeca64a7"
format_stamp: "Formatted at 2026-08-08 14:33:13 on dist-test-slave-w206"
server_key: "2e7e917d53212aa7578eeced5c6057d2"
server_key_iv: "97f645ef8408d47e5ec9b1ab1d7631e3"
server_key_version: "encryptionkey@0"
I20260808 14:33:13.951426 30220 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:33:13.953922 30236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:33:13.955332 30220 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:33:13.955511 30220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "48bee00512c34031ab06ac68eeca64a7"
format_stamp: "Formatted at 2026-08-08 14:33:13 on dist-test-slave-w206"
server_key: "2e7e917d53212aa7578eeced5c6057d2"
server_key_iv: "97f645ef8408d47e5ec9b1ab1d7631e3"
server_key_version: "encryptionkey@0"
I20260808 14:33:13.955643 30220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:33:13.974898 30220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:33:13.978451 30220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:33:13.978673 30220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:33:13.986552 30220 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40719
I20260808 14:33:13.986622 30293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40719 every 8 connection(s)
I20260808 14:33:13.987703 30220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:33:13.990608 30294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:33:13.991799 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 30220
I20260808 14:33:13.991945 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:33:13.992329 7669 external_mini_cluster.cc:1428] Setting key 0454bb57790b008d7da4c6c7764a7df8
I20260808 14:33:13.997085 30294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48bee00512c34031ab06ac68eeca64a7: Bootstrap starting.
I20260808 14:33:13.999387 30294 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48bee00512c34031ab06ac68eeca64a7: Neither blocks nor log segments found. Creating new log.
I20260808 14:33:14.000232 30294 log.cc:824] T 00000000000000000000000000000000 P 48bee00512c34031ab06ac68eeca64a7: Log is configured to *not* fsync() on all Append() calls
I20260808 14:33:14.001835 30297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:13.993849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:57825 (local address 127.0.0.1:40719)
0808 14:33:13.994299 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:13.994315 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:13.994351 (+ 36us) server_negotiation.cc:407] Connection header received
0808 14:33:13.995104 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:13.995124 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:13.995475 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:13.995822 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:13.996252 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:13.997256 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:13.998161 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:13.998636 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:13.998807 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:13.999851 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:13.999856 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:13.999876 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:14.000274 (+ 398us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:14.000373 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:14.000415 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:14.000521 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:14.000742 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 14:33:14.000924 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":142,"threads_started":1}
I20260808 14:33:14.002470 30294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48bee00512c34031ab06ac68eeca64a7: No bootstrap required, opened a new log
I20260808 14:33:14.005039 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:33:14.006654 30294 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48bee00512c34031ab06ac68eeca64a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48bee00512c34031ab06ac68eeca64a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40719 } }
I20260808 14:33:14.006922 30294 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48bee00512c34031ab06ac68eeca64a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:33:14.007036 30294 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48bee00512c34031ab06ac68eeca64a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48bee00512c34031ab06ac68eeca64a7, State: Initialized, Role: FOLLOWER
I20260808 14:33:14.007583 30294 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48bee00512c34031ab06ac68eeca64a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48bee00512c34031ab06ac68eeca64a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40719 } }
I20260808 14:33:14.007776 30294 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48bee00512c34031ab06ac68eeca64a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:33:14.007887 30294 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48bee00512c34031ab06ac68eeca64a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:33:14.008013 30294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48bee00512c34031ab06ac68eeca64a7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:33:14.009186 30294 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48bee00512c34031ab06ac68eeca64a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48bee00512c34031ab06ac68eeca64a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40719 } }
I20260808 14:33:14.009550 30294 leader_election.cc:302] T 00000000000000000000000000000000 P 48bee00512c34031ab06ac68eeca64a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48bee00512c34031ab06ac68eeca64a7; no voters:
I20260808 14:33:14.009923 30294 leader_election.cc:288] T 00000000000000000000000000000000 P 48bee00512c34031ab06ac68eeca64a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:33:14.010030 30305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48bee00512c34031ab06ac68eeca64a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:33:14.010272 30305 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48bee00512c34031ab06ac68eeca64a7 [term 1 LEADER]: Becoming Leader. State: Replica: 48bee00512c34031ab06ac68eeca64a7, State: Running, Role: LEADER
I20260808 14:33:14.010670 30305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48bee00512c34031ab06ac68eeca64a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48bee00512c34031ab06ac68eeca64a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40719 } }
I20260808 14:33:14.011410 30294 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48bee00512c34031ab06ac68eeca64a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:33:14.011807 30297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:14.007180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:35239 (local address 127.0.0.1:40719)
0808 14:33:14.007313 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:14.007317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:14.007518 (+ 201us) server_negotiation.cc:407] Connection header received
0808 14:33:14.007610 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:14.007613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:14.007660 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:14.007738 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:14.008088 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:14.008816 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:14.009724 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:14.010067 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:14.010175 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:14.011254 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:14.011257 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:14.011260 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:14.011275 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:14.011325 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:14.011341 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:14.011431 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:14.011625 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 14:33:14.011660 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:33:14.012496 30307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48bee00512c34031ab06ac68eeca64a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48bee00512c34031ab06ac68eeca64a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48bee00512c34031ab06ac68eeca64a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40719 } } }
I20260808 14:33:14.012640 30307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48bee00512c34031ab06ac68eeca64a7 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:14.013051 30308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48bee00512c34031ab06ac68eeca64a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48bee00512c34031ab06ac68eeca64a7. Latest consensus state: current_term: 1 leader_uuid: "48bee00512c34031ab06ac68eeca64a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48bee00512c34031ab06ac68eeca64a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40719 } } }
I20260808 14:33:14.013173 30308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48bee00512c34031ab06ac68eeca64a7 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:14.013464 30315 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:33:14.016722 30315 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:33:14.024537 30315 catalog_manager.cc:1347] Generated new cluster ID: fe51bf244242400e9a9cb2159147df1b
I20260808 14:33:14.024627 30315 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:33:14.043125 30315 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:33:14.044129 30315 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:33:14.050572 30315 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48bee00512c34031ab06ac68eeca64a7: Generated new TSK 0
I20260808 14:33:14.051292 30315 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:33:14.055711 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 30220
W20260808 14:33:14.063161 30204 connection.cc:535] client connection to 127.0.0.1:40719 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40719): Connection reset by peer (error 104)
2026-08-08T14:33:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:33:14 dist-test-slave-w206 krb5kdc[30327](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:33:14 dist-test-slave-w206 krb5kdc[30327](info): set up 2 sockets
Aug 08 14:33:14 dist-test-slave-w206 krb5kdc[30327](info): commencing operation
krb5kdc: starting...
W20260808 14:33:15.127614 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:33:15 dist-test-slave-w206 krb5kdc[30327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:33:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:33:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:33:15.260445 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39615
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:33:15.375267 30340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:33:15.375578 30340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:33:15.375658 30340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:33:15.379803 30340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:33:15.379900 30340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:33:15.379976 30340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:33:15.380039 30340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:33:15.380079 30340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:33:15.385969 30340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39615
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:33:15.387161 30340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:33:15.388228 30340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:33:15.397079 30349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:15.397161 30340 server_base.cc:1034] running on GCE node
W20260808 14:33:15.397075 30347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:15.397464 30346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:15.398023 30340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:33:15.399245 30340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:33:15.400414 30340 hybrid_clock.cc:648] HybridClock initialized: now 1786199595400373 us; error 52 us; skew 500 ppm
Aug 08 14:33:15 dist-test-slave-w206 krb5kdc[30327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199595, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:33:15.403224 30340 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:33:15.404058 30340 webserver.cc:460] Webserver started at http://127.0.0.1:35907/ using document root <none> and password file <none>
I20260808 14:33:15.404574 30340 fs_manager.cc:359] Metadata directory not provided
I20260808 14:33:15.404629 30340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:33:15.404793 30340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:33:15.406854 30340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "ae4be6c5651f41b783cd4e207d5909ee"
format_stamp: "Formatted at 2026-08-08 14:33:15 on dist-test-slave-w206"
server_key: "bd8de0d274a4c2cf1af0b193c3a9b7ab"
server_key_iv: "e148c12dc32a4c4ccbb40a94f4840c85"
server_key_version: "encryptionkey@0"
I20260808 14:33:15.407518 30340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "ae4be6c5651f41b783cd4e207d5909ee"
format_stamp: "Formatted at 2026-08-08 14:33:15 on dist-test-slave-w206"
server_key: "bd8de0d274a4c2cf1af0b193c3a9b7ab"
server_key_iv: "e148c12dc32a4c4ccbb40a94f4840c85"
server_key_version: "encryptionkey@0"
I20260808 14:33:15.411383 30340 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:33:15.413841 30357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:33:15.415242 30340 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:33:15.415392 30340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "ae4be6c5651f41b783cd4e207d5909ee"
format_stamp: "Formatted at 2026-08-08 14:33:15 on dist-test-slave-w206"
server_key: "bd8de0d274a4c2cf1af0b193c3a9b7ab"
server_key_iv: "e148c12dc32a4c4ccbb40a94f4840c85"
server_key_version: "encryptionkey@0"
I20260808 14:33:15.415518 30340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:33:15.420430 30340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:33:15.423574 30340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:33:15.423779 30340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:33:15.430951 30340 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39615
I20260808 14:33:15.431061 30413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39615 every 8 connection(s)
I20260808 14:33:15.432019 30340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:33:15.434806 30414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:33:15.437816 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 30340
I20260808 14:33:15.437933 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:33:15.438234 7669 external_mini_cluster.cc:1428] Setting key 97a7caf85e8ee8e530da9bb9e9839d81
I20260808 14:33:15.441085 30414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae4be6c5651f41b783cd4e207d5909ee: Bootstrap starting.
I20260808 14:33:15.443951 30414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae4be6c5651f41b783cd4e207d5909ee: Neither blocks nor log segments found. Creating new log.
I20260808 14:33:15.444101 30418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:15.439765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:36007 (local address 127.0.0.1:39615)
0808 14:33:15.440219 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:15.440240 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:15.440283 (+ 43us) server_negotiation.cc:407] Connection header received
0808 14:33:15.441112 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:15.441135 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:15.441510 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:15.441865 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:15.442655 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:15.442661 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:15.442676 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:15.442689 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:15.442754 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:15.442781 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:15.442920 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:15.443012 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 14:33:15.443166 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":125,"threads_started":1}
I20260808 14:33:15.444790 30414 log.cc:824] T 00000000000000000000000000000000 P ae4be6c5651f41b783cd4e207d5909ee: Log is configured to *not* fsync() on all Append() calls
I20260808 14:33:15.447268 30414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae4be6c5651f41b783cd4e207d5909ee: No bootstrap required, opened a new log
I20260808 14:33:15.447881 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:33:15.451428 30414 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae4be6c5651f41b783cd4e207d5909ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4be6c5651f41b783cd4e207d5909ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39615 } }
I20260808 14:33:15.451766 30414 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae4be6c5651f41b783cd4e207d5909ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:33:15.451815 30418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:15.450038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:37821 (local address 127.0.0.1:39615)
0808 14:33:15.450182 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:15.450189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:15.450422 (+ 233us) server_negotiation.cc:407] Connection header received
0808 14:33:15.450493 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:15.450496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:15.450535 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:15.450622 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:15.451430 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:15.451433 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:15.451437 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:15.451453 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:15.451482 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:15.451496 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:15.451590 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:15.451645 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 14:33:15.451684 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 14:33:15.451831 30414 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae4be6c5651f41b783cd4e207d5909ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae4be6c5651f41b783cd4e207d5909ee, State: Initialized, Role: FOLLOWER
I20260808 14:33:15.452291 30414 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae4be6c5651f41b783cd4e207d5909ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4be6c5651f41b783cd4e207d5909ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39615 } }
I20260808 14:33:15.452425 30414 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae4be6c5651f41b783cd4e207d5909ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:33:15.452500 30414 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae4be6c5651f41b783cd4e207d5909ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:33:15.452630 30414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae4be6c5651f41b783cd4e207d5909ee [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:33:15.453701 30414 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae4be6c5651f41b783cd4e207d5909ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4be6c5651f41b783cd4e207d5909ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39615 } }
I20260808 14:33:15.454070 30414 leader_election.cc:302] T 00000000000000000000000000000000 P ae4be6c5651f41b783cd4e207d5909ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae4be6c5651f41b783cd4e207d5909ee; no voters:
I20260808 14:33:15.454396 30414 leader_election.cc:288] T 00000000000000000000000000000000 P ae4be6c5651f41b783cd4e207d5909ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:33:15.454571 30426 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae4be6c5651f41b783cd4e207d5909ee [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:33:15.454826 30426 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae4be6c5651f41b783cd4e207d5909ee [term 1 LEADER]: Becoming Leader. State: Replica: ae4be6c5651f41b783cd4e207d5909ee, State: Running, Role: LEADER
I20260808 14:33:15.455159 30426 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae4be6c5651f41b783cd4e207d5909ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4be6c5651f41b783cd4e207d5909ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39615 } }
I20260808 14:33:15.455703 30414 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae4be6c5651f41b783cd4e207d5909ee [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:33:15.456957 30428 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae4be6c5651f41b783cd4e207d5909ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae4be6c5651f41b783cd4e207d5909ee. Latest consensus state: current_term: 1 leader_uuid: "ae4be6c5651f41b783cd4e207d5909ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4be6c5651f41b783cd4e207d5909ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39615 } } }
I20260808 14:33:15.457086 30428 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae4be6c5651f41b783cd4e207d5909ee [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:15.457425 30427 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae4be6c5651f41b783cd4e207d5909ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae4be6c5651f41b783cd4e207d5909ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4be6c5651f41b783cd4e207d5909ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39615 } } }
I20260808 14:33:15.457576 30427 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae4be6c5651f41b783cd4e207d5909ee [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:15.457494 30435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:33:15.460597 30435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:33:15.467144 30435 catalog_manager.cc:1347] Generated new cluster ID: 6ac4e6ae21ff4c68beec62e705e69100
I20260808 14:33:15.467232 30435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:33:15.490908 30435 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:33:15.491276 30435 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:33:15.497579 30435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae4be6c5651f41b783cd4e207d5909ee: Generated new TSK 0
I20260808 14:33:15.498283 30435 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:33:15.508553 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 30340
2026-08-08T14:33:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1453 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:33:15 dist-test-slave-w206 krb5kdc[30447](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:33:15 dist-test-slave-w206 krb5kdc[30447](info): set up 2 sockets
Aug 08 14:33:15 dist-test-slave-w206 krb5kdc[30447](info): commencing operation
krb5kdc: starting...
W20260808 14:33:16.628026 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:33:16 dist-test-slave-w206 krb5kdc[30447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:33:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:33:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:33:16.762359 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:34143
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:40715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:34143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:33:16.878675 30461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:33:16.879024 30461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:33:16.879112 30461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:33:16.883322 30461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:33:16.883383 30461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:33:16.883404 30461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:33:16.883420 30461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:33:16.883436 30461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:33:16.888851 30461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:40715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:34143
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:34143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:33:16.889997 30461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:33:16.891070 30461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:33:16.900058 30469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:16.900064 30471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:16.900064 30468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:16.900560 30461 server_base.cc:1034] running on GCE node
I20260808 14:33:16.901181 30461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:33:16.902366 30461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:33:16.903558 30461 hybrid_clock.cc:648] HybridClock initialized: now 1786199596903534 us; error 40 us; skew 500 ppm
Aug 08 14:33:16 dist-test-slave-w206 krb5kdc[30447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199596, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:33:16.913764 30461 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:33:16.914855 30461 webserver.cc:460] Webserver started at http://127.7.125.126:38951/ using document root <none> and password file <none>
I20260808 14:33:16.915521 30461 fs_manager.cc:359] Metadata directory not provided
I20260808 14:33:16.915627 30461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:33:16.915887 30461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:33:16.918056 30461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "4adab5778ce344fcad9a3c6ab79ac936"
format_stamp: "Formatted at 2026-08-08 14:33:16 on dist-test-slave-w206"
server_key: "f40841438e0acdb280e52fbced296215"
server_key_iv: "5d4d27c156f8865866f140a0880e705b"
server_key_version: "encryptionkey@0"
I20260808 14:33:16.918671 30461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "4adab5778ce344fcad9a3c6ab79ac936"
format_stamp: "Formatted at 2026-08-08 14:33:16 on dist-test-slave-w206"
server_key: "f40841438e0acdb280e52fbced296215"
server_key_iv: "5d4d27c156f8865866f140a0880e705b"
server_key_version: "encryptionkey@0"
I20260808 14:33:16.922623 30461 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 14:33:16.925175 30478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:33:16.926440 30461 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:33:16.926591 30461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "4adab5778ce344fcad9a3c6ab79ac936"
format_stamp: "Formatted at 2026-08-08 14:33:16 on dist-test-slave-w206"
server_key: "f40841438e0acdb280e52fbced296215"
server_key_iv: "5d4d27c156f8865866f140a0880e705b"
server_key_version: "encryptionkey@0"
I20260808 14:33:16.926712 30461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:33:16.948879 30461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:33:16.952037 30461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:33:16.952278 30461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:33:16.959712 30461 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:34143
I20260808 14:33:16.959739 30530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:34143 every 8 connection(s)
I20260808 14:33:16.960695 30461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:33:16.963495 30531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:33:16.969113 30531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4adab5778ce344fcad9a3c6ab79ac936: Bootstrap starting.
I20260808 14:33:16.970763 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 30461
I20260808 14:33:16.970932 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:33:16.971251 7669 external_mini_cluster.cc:1428] Setting key de226b69a420e798aacf0596c703483f
I20260808 14:33:16.971971 30531 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4adab5778ce344fcad9a3c6ab79ac936: Neither blocks nor log segments found. Creating new log.
I20260808 14:33:16.972791 30531 log.cc:824] T 00000000000000000000000000000000 P 4adab5778ce344fcad9a3c6ab79ac936: Log is configured to *not* fsync() on all Append() calls
I20260808 14:33:16.975306 30531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4adab5778ce344fcad9a3c6ab79ac936: No bootstrap required, opened a new log
I20260808 14:33:16.979879 30531 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4adab5778ce344fcad9a3c6ab79ac936 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4adab5778ce344fcad9a3c6ab79ac936" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 34143 } }
I20260808 14:33:16.980126 30531 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4adab5778ce344fcad9a3c6ab79ac936 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:33:16.980193 30531 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4adab5778ce344fcad9a3c6ab79ac936 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4adab5778ce344fcad9a3c6ab79ac936, State: Initialized, Role: FOLLOWER
I20260808 14:33:16.980687 30531 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4adab5778ce344fcad9a3c6ab79ac936 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4adab5778ce344fcad9a3c6ab79ac936" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 34143 } }
I20260808 14:33:16.980813 30531 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4adab5778ce344fcad9a3c6ab79ac936 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:33:16.980901 30531 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4adab5778ce344fcad9a3c6ab79ac936 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:33:16.980997 30531 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4adab5778ce344fcad9a3c6ab79ac936 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 14:33:16 dist-test-slave-w206 krb5kdc[30447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:33:16.982020 30531 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4adab5778ce344fcad9a3c6ab79ac936 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4adab5778ce344fcad9a3c6ab79ac936" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 34143 } }
I20260808 14:33:16.982424 30531 leader_election.cc:302] T 00000000000000000000000000000000 P 4adab5778ce344fcad9a3c6ab79ac936 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4adab5778ce344fcad9a3c6ab79ac936; no voters:
I20260808 14:33:16.982813 30531 leader_election.cc:288] T 00000000000000000000000000000000 P 4adab5778ce344fcad9a3c6ab79ac936 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:33:16.982918 30536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4adab5778ce344fcad9a3c6ab79ac936 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:33:16.983120 30536 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4adab5778ce344fcad9a3c6ab79ac936 [term 1 LEADER]: Becoming Leader. State: Replica: 4adab5778ce344fcad9a3c6ab79ac936, State: Running, Role: LEADER
I20260808 14:33:16.983414 30536 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4adab5778ce344fcad9a3c6ab79ac936 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4adab5778ce344fcad9a3c6ab79ac936" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 34143 } }
I20260808 14:33:16.984167 30531 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4adab5778ce344fcad9a3c6ab79ac936 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:33:16.984611 30536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4adab5778ce344fcad9a3c6ab79ac936 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4adab5778ce344fcad9a3c6ab79ac936. Latest consensus state: current_term: 1 leader_uuid: "4adab5778ce344fcad9a3c6ab79ac936" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4adab5778ce344fcad9a3c6ab79ac936" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 34143 } } }
I20260808 14:33:16.984764 30536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4adab5778ce344fcad9a3c6ab79ac936 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:16.984612 30537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4adab5778ce344fcad9a3c6ab79ac936 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4adab5778ce344fcad9a3c6ab79ac936" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4adab5778ce344fcad9a3c6ab79ac936" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 34143 } } }
I20260808 14:33:16.985082 30537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4adab5778ce344fcad9a3c6ab79ac936 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:16.985663 30542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:33:16.988281 30534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:16.972464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51917 (local address 127.7.125.126:34143)
0808 14:33:16.972941 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:16.972967 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:16.973002 (+ 35us) server_negotiation.cc:407] Connection header received
0808 14:33:16.973956 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:16.973985 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:16.974505 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:16.974877 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:16.975982 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:16.977458 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:16.978374 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:16.979157 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:16.979319 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:16.982086 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:16.982095 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:16.982116 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:16.982149 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:16.984279 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:16.985053 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:16.985058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:16.985066 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:16.985296 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:16.985720 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:16.985725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:16.985729 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:16.986244 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:16.986470 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:16.987041 (+ 571us) server_negotiation.cc:299] Negotiation successful
0808 14:33:16.987302 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":169,"threads_started":1}
I20260808 14:33:16.988750 30542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:33:16.991591 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:33:16.997031 30542 catalog_manager.cc:1347] Generated new cluster ID: f6d6d076a7d44b2ab0d749dbf0705cd4
I20260808 14:33:16.997098 30542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:33:17.005034 30534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:16.995447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48911 (local address 127.7.125.126:34143)
0808 14:33:16.995570 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:16.995574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:16.995846 (+ 272us) server_negotiation.cc:407] Connection header received
0808 14:33:16.996004 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:16.996007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:16.996055 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:16.996140 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:16.997036 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:16.997857 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:16.999105 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:16.999457 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:16.999558 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:17.001389 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:17.001392 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:17.001395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:17.001421 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:17.003128 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:17.003682 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:17.003686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:17.003688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:17.003766 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:17.004125 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:17.004128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:17.004130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:17.004326 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:17.004473 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:17.004763 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 14:33:17.004872 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 14:33:17.011016 30542 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:33:17.011921 30542 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:33:17.020043 30542 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4adab5778ce344fcad9a3c6ab79ac936: Generated new TSK 0
I20260808 14:33:17.020792 30542 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:33:17.026587 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 30461
W20260808 14:33:17.033679 30445 connection.cc:535] client connection to 127.7.125.126:34143 recv error: Network error: failed to read from TLS socket (remote: 127.7.125.126:34143): Connection reset by peer (error 104)
2026-08-08T14:33:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1518 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:33:17 dist-test-slave-w206 krb5kdc[30568](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:33:17 dist-test-slave-w206 krb5kdc[30568](info): set up 2 sockets
Aug 08 14:33:17 dist-test-slave-w206 krb5kdc[30568](info): commencing operation
krb5kdc: starting...
W20260808 14:33:18.096243 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:33:18 dist-test-slave-w206 krb5kdc[30568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:33:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:33:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:33:18.231303 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:41433
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:41433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:33:18.351011 30581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:33:18.351339 30581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:33:18.351418 30581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:33:18.355578 30581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:33:18.355643 30581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:33:18.355662 30581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:33:18.355679 30581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:33:18.355695 30581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:33:18.361303 30581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:41433
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:41433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:33:18.362393 30581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:33:18.363498 30581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:33:18.372378 30581 server_base.cc:1034] running on GCE node
W20260808 14:33:18.372277 30587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:18.372298 30589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:18.372335 30586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:18.373090 30581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:33:18.374172 30581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:33:18.375335 30581 hybrid_clock.cc:648] HybridClock initialized: now 1786199598375317 us; error 31 us; skew 500 ppm
Aug 08 14:33:18 dist-test-slave-w206 krb5kdc[30568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199598, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:33:18.378140 30581 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:33:18.379230 30581 webserver.cc:460] Webserver started at http://127.7.125.126:45805/ using document root <none> and password file <none>
I20260808 14:33:18.379807 30581 fs_manager.cc:359] Metadata directory not provided
I20260808 14:33:18.379910 30581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:33:18.380173 30581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:33:18.382226 30581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "f4c2da25248f494aa6e07d68bb027f4a"
format_stamp: "Formatted at 2026-08-08 14:33:18 on dist-test-slave-w206"
server_key: "86bcf3f7d21e82177d909bf4b5018e66"
server_key_iv: "9a4e581f6998b33482178b34630e147c"
server_key_version: "encryptionkey@0"
I20260808 14:33:18.382807 30581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "f4c2da25248f494aa6e07d68bb027f4a"
format_stamp: "Formatted at 2026-08-08 14:33:18 on dist-test-slave-w206"
server_key: "86bcf3f7d21e82177d909bf4b5018e66"
server_key_iv: "9a4e581f6998b33482178b34630e147c"
server_key_version: "encryptionkey@0"
I20260808 14:33:18.386682 30581 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 14:33:18.389295 30596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:33:18.390604 30581 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:33:18.390755 30581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "f4c2da25248f494aa6e07d68bb027f4a"
format_stamp: "Formatted at 2026-08-08 14:33:18 on dist-test-slave-w206"
server_key: "86bcf3f7d21e82177d909bf4b5018e66"
server_key_iv: "9a4e581f6998b33482178b34630e147c"
server_key_version: "encryptionkey@0"
I20260808 14:33:18.390877 30581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:33:18.408238 30581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:33:18.412153 30581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:33:18.412392 30581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:33:18.419848 30581 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:41433
I20260808 14:33:18.419884 30651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:41433 every 8 connection(s)
I20260808 14:33:18.420847 30581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:33:18.423640 30652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:33:18.429277 30652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a: Bootstrap starting.
I20260808 14:33:18.429646 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 30581
I20260808 14:33:18.429796 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:33:18.430101 7669 external_mini_cluster.cc:1428] Setting key ac96d9ddf834a83d57bab1de9f2ba44c
I20260808 14:33:18.432073 30652 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a: Neither blocks nor log segments found. Creating new log.
I20260808 14:33:18.432819 30652 log.cc:824] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a: Log is configured to *not* fsync() on all Append() calls
I20260808 14:33:18.434917 30652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a: No bootstrap required, opened a new log
I20260808 14:33:18.438469 30652 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c2da25248f494aa6e07d68bb027f4a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 41433 } }
I20260808 14:33:18.438794 30652 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:33:18.438885 30652 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c2da25248f494aa6e07d68bb027f4a, State: Initialized, Role: FOLLOWER
I20260808 14:33:18.439419 30652 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c2da25248f494aa6e07d68bb027f4a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 41433 } }
I20260808 14:33:18.439491 30655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:18.431532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57891 (local address 127.7.125.126:41433)
0808 14:33:18.431987 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:18.432016 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:18.432050 (+ 34us) server_negotiation.cc:407] Connection header received
0808 14:33:18.432750 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:18.432767 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:18.433235 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:18.433627 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:18.433971 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:18.434926 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:18.435920 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:18.436406 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:18.436589 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:18.437546 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:18.437551 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:18.437565 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:18.437967 (+ 402us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:18.438025 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:18.438049 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:18.438199 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:18.438407 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 14:33:18.438551 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":146,"threads_started":1}
I20260808 14:33:18.439595 30652 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:33:18.439677 30652 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:33:18.439822 30652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:33:18.440999 30652 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c2da25248f494aa6e07d68bb027f4a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 41433 } }
I20260808 14:33:18.441397 30652 leader_election.cc:302] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4c2da25248f494aa6e07d68bb027f4a; no voters:
I20260808 14:33:18.441769 30652 leader_election.cc:288] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:33:18.441871 30657 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:33:18.442170 30657 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a [term 1 LEADER]: Becoming Leader. State: Replica: f4c2da25248f494aa6e07d68bb027f4a, State: Running, Role: LEADER
I20260808 14:33:18.442571 30657 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c2da25248f494aa6e07d68bb027f4a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 41433 } }
I20260808 14:33:18.442798 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:33:18.443228 30652 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:33:18.445353 30658 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4c2da25248f494aa6e07d68bb027f4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c2da25248f494aa6e07d68bb027f4a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 41433 } } }
I20260808 14:33:18.445482 30658 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:18.445803 30659 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4c2da25248f494aa6e07d68bb027f4a. Latest consensus state: current_term: 1 leader_uuid: "f4c2da25248f494aa6e07d68bb027f4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c2da25248f494aa6e07d68bb027f4a" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 41433 } } }
I20260808 14:33:18.446153 30659 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a [sys.catalog]: This master's current role is: LEADER
W20260808 14:33:18.448979 30677 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:33:18.449062 30677 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:33:18.449139 30679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:33:18.450030 30679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:33:18.451620 30655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:18.446338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50325 (local address 127.7.125.126:41433)
0808 14:33:18.446488 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:18.446492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:18.446748 (+ 256us) server_negotiation.cc:407] Connection header received
0808 14:33:18.446861 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:18.446864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:18.446903 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:18.447016 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:18.447343 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:18.448090 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:18.449417 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:18.449741 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:18.449817 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:18.451079 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:18.451083 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:18.451086 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:18.451101 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:18.451130 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:18.451143 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:18.451204 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:18.451447 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 14:33:18.451487 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 14:33:18.456543 30679 catalog_manager.cc:1347] Generated new cluster ID: 66bfa04f82144abfb57e6505d7d897ba
I20260808 14:33:18.456627 30679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:33:18.467604 30679 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:33:18.468578 30679 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:33:18.477527 30679 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4c2da25248f494aa6e07d68bb027f4a: Generated new TSK 0
I20260808 14:33:18.478302 30679 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:33:18.486455 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 30581
W20260808 14:33:18.493119 30564 connection.cc:535] client connection to 127.7.125.126:41433 recv error: Network error: failed to read from TLS socket (remote: 127.7.125.126:41433): Connection reset by peer (error 104)
2026-08-08T14:33:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1458 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:33:18 dist-test-slave-w206 krb5kdc[30685](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:33:18 dist-test-slave-w206 krb5kdc[30685](info): set up 2 sockets
Aug 08 14:33:18 dist-test-slave-w206 krb5kdc[30685](info): commencing operation
krb5kdc: starting...
W20260808 14:33:19.560918 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:33:19 dist-test-slave-w206 krb5kdc[30685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:33:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:33:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:33:19.693567 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:42941
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:43851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:42941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:33:19.808916 30699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:33:19.809239 30699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:33:19.809324 30699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:33:19.813426 30699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:33:19.813544 30699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:33:19.813581 30699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:33:19.813619 30699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:33:19.813649 30699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:33:19.819207 30699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:43851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:42941
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:42941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:33:19.820421 30699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:33:19.821533 30699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:33:19.830905 30704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:19.831012 30705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:19.830904 30708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:19.831228 30699 server_base.cc:1034] running on GCE node
I20260808 14:33:19.831812 30699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:33:19.833041 30699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:33:19.834247 30699 hybrid_clock.cc:648] HybridClock initialized: now 1786199599834225 us; error 38 us; skew 500 ppm
Aug 08 14:33:19 dist-test-slave-w206 krb5kdc[30685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199599, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:33:19.837177 30699 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:33:19.838160 30699 webserver.cc:460] Webserver started at http://127.7.125.126:42401/ using document root <none> and password file <none>
I20260808 14:33:19.838742 30699 fs_manager.cc:359] Metadata directory not provided
I20260808 14:33:19.838836 30699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:33:19.839190 30699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:33:19.841331 30699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "4d961d5294a14a4badee39b5ad983746"
format_stamp: "Formatted at 2026-08-08 14:33:19 on dist-test-slave-w206"
server_key: "a0bcb1e828a33064ac608895e0442d85"
server_key_iv: "3226fccf14e52636145f0c57e691a352"
server_key_version: "encryptionkey@0"
I20260808 14:33:19.841918 30699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "4d961d5294a14a4badee39b5ad983746"
format_stamp: "Formatted at 2026-08-08 14:33:19 on dist-test-slave-w206"
server_key: "a0bcb1e828a33064ac608895e0442d85"
server_key_iv: "3226fccf14e52636145f0c57e691a352"
server_key_version: "encryptionkey@0"
I20260808 14:33:19.845917 30699 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:33:19.848496 30715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:33:19.849874 30699 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:33:19.850018 30699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "4d961d5294a14a4badee39b5ad983746"
format_stamp: "Formatted at 2026-08-08 14:33:19 on dist-test-slave-w206"
server_key: "a0bcb1e828a33064ac608895e0442d85"
server_key_iv: "3226fccf14e52636145f0c57e691a352"
server_key_version: "encryptionkey@0"
I20260808 14:33:19.850139 30699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:33:19.854852 30699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:33:19.858237 30699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:33:19.858446 30699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:33:19.865937 30699 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:42941
I20260808 14:33:19.865963 30771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:42941 every 8 connection(s)
I20260808 14:33:19.866878 30699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:33:19.869670 30772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:33:19.870929 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 30699
I20260808 14:33:19.871126 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:33:19.871493 7669 external_mini_cluster.cc:1428] Setting key 8a969bc202891a4e864aa2bfca6e07af
I20260808 14:33:19.876680 30772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d961d5294a14a4badee39b5ad983746: Bootstrap starting.
I20260808 14:33:19.877071 30775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:19.873054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41105 (local address 127.7.125.126:42941)
0808 14:33:19.873554 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:19.873570 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:19.873608 (+ 38us) server_negotiation.cc:407] Connection header received
0808 14:33:19.874262 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:19.874279 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:19.874582 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:19.874929 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:19.875826 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:19.875830 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:19.875842 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:19.875852 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:19.875907 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:19.875927 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:19.876082 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:19.876151 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 14:33:19.876312 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":167,"threads_started":1}
I20260808 14:33:19.879359 30772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d961d5294a14a4badee39b5ad983746: Neither blocks nor log segments found. Creating new log.
I20260808 14:33:19.880034 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:33:19.880309 30772 log.cc:824] T 00000000000000000000000000000000 P 4d961d5294a14a4badee39b5ad983746: Log is configured to *not* fsync() on all Append() calls
I20260808 14:33:19.882419 30772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d961d5294a14a4badee39b5ad983746: No bootstrap required, opened a new log
I20260808 14:33:19.884050 30775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:19.882122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51849 (local address 127.7.125.126:42941)
0808 14:33:19.882242 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:19.882246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:19.882645 (+ 399us) server_negotiation.cc:407] Connection header received
0808 14:33:19.882796 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:19.882799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:19.882840 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:19.882928 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:19.883654 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:19.883658 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:19.883660 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:19.883672 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:19.883699 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:19.883717 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:19.883803 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:19.883901 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 14:33:19.883938 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 14:33:19.886550 30772 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d961d5294a14a4badee39b5ad983746 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d961d5294a14a4badee39b5ad983746" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 42941 } }
I20260808 14:33:19.886780 30772 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d961d5294a14a4badee39b5ad983746 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:33:19.886840 30772 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d961d5294a14a4badee39b5ad983746 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d961d5294a14a4badee39b5ad983746, State: Initialized, Role: FOLLOWER
I20260808 14:33:19.887326 30772 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d961d5294a14a4badee39b5ad983746 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d961d5294a14a4badee39b5ad983746" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 42941 } }
I20260808 14:33:19.887475 30772 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d961d5294a14a4badee39b5ad983746 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:33:19.887539 30772 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d961d5294a14a4badee39b5ad983746 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:33:19.887676 30772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d961d5294a14a4badee39b5ad983746 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:33:19.888691 30772 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d961d5294a14a4badee39b5ad983746 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d961d5294a14a4badee39b5ad983746" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 42941 } }
I20260808 14:33:19.889048 30772 leader_election.cc:302] T 00000000000000000000000000000000 P 4d961d5294a14a4badee39b5ad983746 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d961d5294a14a4badee39b5ad983746; no voters:
I20260808 14:33:19.889355 30772 leader_election.cc:288] T 00000000000000000000000000000000 P 4d961d5294a14a4badee39b5ad983746 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:33:19.889516 30782 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d961d5294a14a4badee39b5ad983746 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:33:19.889731 30782 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d961d5294a14a4badee39b5ad983746 [term 1 LEADER]: Becoming Leader. State: Replica: 4d961d5294a14a4badee39b5ad983746, State: Running, Role: LEADER
I20260808 14:33:19.890061 30782 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d961d5294a14a4badee39b5ad983746 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d961d5294a14a4badee39b5ad983746" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 42941 } }
I20260808 14:33:19.890720 30772 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d961d5294a14a4badee39b5ad983746 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:33:19.891921 30783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d961d5294a14a4badee39b5ad983746 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d961d5294a14a4badee39b5ad983746" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d961d5294a14a4badee39b5ad983746" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 42941 } } }
I20260808 14:33:19.892050 30783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d961d5294a14a4badee39b5ad983746 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:19.892458 30791 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:33:19.892438 30784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d961d5294a14a4badee39b5ad983746 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d961d5294a14a4badee39b5ad983746. Latest consensus state: current_term: 1 leader_uuid: "4d961d5294a14a4badee39b5ad983746" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d961d5294a14a4badee39b5ad983746" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 42941 } } }
I20260808 14:33:19.892673 30784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d961d5294a14a4badee39b5ad983746 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:19.895756 30791 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:33:19.901561 30791 catalog_manager.cc:1347] Generated new cluster ID: 0e175746404749bf86b6f453fdc68f1b
I20260808 14:33:19.901639 30791 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:33:19.911443 30791 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:33:19.911778 30791 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:33:19.921368 30791 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d961d5294a14a4badee39b5ad983746: Generated new TSK 0
I20260808 14:33:19.922248 30791 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:33:19.928212 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 30699
2026-08-08T14:33:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1442 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:33:19 dist-test-slave-w206 krb5kdc[30805](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:33:19 dist-test-slave-w206 krb5kdc[30805](info): set up 2 sockets
Aug 08 14:33:19 dist-test-slave-w206 krb5kdc[30805](info): commencing operation
krb5kdc: starting...
W20260808 14:33:21.008656 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:33:21 dist-test-slave-w206 krb5kdc[30805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:33:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:33:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:33:21.141775 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:33123
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:42569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:33123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:33:21.256093 30819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:33:21.256387 30819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:33:21.256433 30819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:33:21.260430 30819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:33:21.260491 30819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:33:21.260511 30819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:33:21.260528 30819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:33:21.260545 30819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:33:21.265957 30819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:42569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:33123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:33123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:33:21.267171 30819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:33:21.268167 30819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:33:21.277288 30819 server_base.cc:1034] running on GCE node
W20260808 14:33:21.277109 30824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:21.277164 30827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:21.277097 30825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:21.278173 30819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:33:21.279443 30819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:33:21.280625 30819 hybrid_clock.cc:648] HybridClock initialized: now 1786199601280605 us; error 31 us; skew 500 ppm
Aug 08 14:33:21 dist-test-slave-w206 krb5kdc[30805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199601, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:33:21.283988 30819 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:33:21.284864 30819 webserver.cc:460] Webserver started at http://127.7.125.126:34975/ using document root <none> and password file <none>
I20260808 14:33:21.285465 30819 fs_manager.cc:359] Metadata directory not provided
I20260808 14:33:21.285583 30819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:33:21.285792 30819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:33:21.287900 30819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "4a4f1e0194fd4628a481c4978e339150"
format_stamp: "Formatted at 2026-08-08 14:33:21 on dist-test-slave-w206"
server_key: "3c94690b312915045fc4f6675e95eeeb"
server_key_iv: "a0d23a444ede445f2b5dab1554530d2d"
server_key_version: "encryptionkey@0"
I20260808 14:33:21.288488 30819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "4a4f1e0194fd4628a481c4978e339150"
format_stamp: "Formatted at 2026-08-08 14:33:21 on dist-test-slave-w206"
server_key: "3c94690b312915045fc4f6675e95eeeb"
server_key_iv: "a0d23a444ede445f2b5dab1554530d2d"
server_key_version: "encryptionkey@0"
I20260808 14:33:21.292240 30819 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:33:21.294888 30834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:33:21.296216 30819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:33:21.296402 30819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "4a4f1e0194fd4628a481c4978e339150"
format_stamp: "Formatted at 2026-08-08 14:33:21 on dist-test-slave-w206"
server_key: "3c94690b312915045fc4f6675e95eeeb"
server_key_iv: "a0d23a444ede445f2b5dab1554530d2d"
server_key_version: "encryptionkey@0"
I20260808 14:33:21.296520 30819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:33:21.327329 30819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:33:21.331089 30819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:33:21.331336 30819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:33:21.338820 30819 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:33123
I20260808 14:33:21.338847 30889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:33123 every 8 connection(s)
I20260808 14:33:21.339998 30819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:33:21.342844 30891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:33:21.348488 30891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a4f1e0194fd4628a481c4978e339150: Bootstrap starting.
I20260808 14:33:21.349470 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 30819
I20260808 14:33:21.349622 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:33:21.349994 7669 external_mini_cluster.cc:1428] Setting key 16be43211b033f2e75eedc4d74bfc4c1
I20260808 14:33:21.351672 30891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a4f1e0194fd4628a481c4978e339150: Neither blocks nor log segments found. Creating new log.
I20260808 14:33:21.352484 30891 log.cc:824] T 00000000000000000000000000000000 P 4a4f1e0194fd4628a481c4978e339150: Log is configured to *not* fsync() on all Append() calls
I20260808 14:33:21.354827 30891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a4f1e0194fd4628a481c4978e339150: No bootstrap required, opened a new log
Aug 08 14:33:21 dist-test-slave-w206 krb5kdc[30805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:33:21.359583 30891 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a4f1e0194fd4628a481c4978e339150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a4f1e0194fd4628a481c4978e339150" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33123 } }
I20260808 14:33:21.359906 30891 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a4f1e0194fd4628a481c4978e339150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:33:21.360013 30891 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a4f1e0194fd4628a481c4978e339150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a4f1e0194fd4628a481c4978e339150, State: Initialized, Role: FOLLOWER
I20260808 14:33:21.360519 30891 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a4f1e0194fd4628a481c4978e339150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a4f1e0194fd4628a481c4978e339150" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33123 } }
I20260808 14:33:21.360678 30891 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a4f1e0194fd4628a481c4978e339150 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:33:21.360754 30891 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a4f1e0194fd4628a481c4978e339150 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:33:21.360865 30891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a4f1e0194fd4628a481c4978e339150 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:33:21.361810 30891 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a4f1e0194fd4628a481c4978e339150 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a4f1e0194fd4628a481c4978e339150" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33123 } }
I20260808 14:33:21.362175 30891 leader_election.cc:302] T 00000000000000000000000000000000 P 4a4f1e0194fd4628a481c4978e339150 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a4f1e0194fd4628a481c4978e339150; no voters:
I20260808 14:33:21.362529 30891 leader_election.cc:288] T 00000000000000000000000000000000 P 4a4f1e0194fd4628a481c4978e339150 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:33:21.362617 30896 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a4f1e0194fd4628a481c4978e339150 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:33:21.362895 30896 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a4f1e0194fd4628a481c4978e339150 [term 1 LEADER]: Becoming Leader. State: Replica: 4a4f1e0194fd4628a481c4978e339150, State: Running, Role: LEADER
I20260808 14:33:21.363325 30896 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a4f1e0194fd4628a481c4978e339150 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a4f1e0194fd4628a481c4978e339150" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33123 } }
I20260808 14:33:21.364046 30891 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a4f1e0194fd4628a481c4978e339150 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:33:21.365119 30897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a4f1e0194fd4628a481c4978e339150 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a4f1e0194fd4628a481c4978e339150" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a4f1e0194fd4628a481c4978e339150" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33123 } } }
I20260808 14:33:21.365094 30898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a4f1e0194fd4628a481c4978e339150 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a4f1e0194fd4628a481c4978e339150. Latest consensus state: current_term: 1 leader_uuid: "4a4f1e0194fd4628a481c4978e339150" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a4f1e0194fd4628a481c4978e339150" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33123 } } }
I20260808 14:33:21.365239 30897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a4f1e0194fd4628a481c4978e339150 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:21.365239 30898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a4f1e0194fd4628a481c4978e339150 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:21.365768 30905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:33:21.366454 30894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:21.351241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45813 (local address 127.7.125.126:33123)
0808 14:33:21.351712 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:21.351751 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:21.351799 (+ 48us) server_negotiation.cc:407] Connection header received
0808 14:33:21.352506 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:21.352524 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:21.352983 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:21.353391 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:21.354319 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:21.355448 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:21.356659 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:21.357150 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:21.357280 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:21.360403 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:21.360408 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:21.360424 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:21.360455 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:21.362330 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:21.362864 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:21.362870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:21.362877 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:21.363084 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:21.363432 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:21.363435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:21.363437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:21.363962 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:21.364166 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:21.364983 (+ 817us) server_negotiation.cc:299] Negotiation successful
0808 14:33:21.365354 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":148,"threads_started":1}
I20260808 14:33:21.369560 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:33:21.370565 30905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:33:21.376786 30905 catalog_manager.cc:1347] Generated new cluster ID: 78bb6ad1a90a49729c67d1a65a535570
I20260808 14:33:21.376899 30905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:33:21.380828 30894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:21.371686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55615 (local address 127.7.125.126:33123)
0808 14:33:21.371805 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:21.371810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:21.372046 (+ 236us) server_negotiation.cc:407] Connection header received
0808 14:33:21.372185 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:21.372188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:21.372231 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:21.372322 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:21.373155 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:21.373967 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:21.374937 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:21.375381 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:21.375486 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:21.377222 (+ 1736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:21.377225 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:21.377228 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:21.377252 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:21.378947 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:21.379501 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:21.379505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:21.379507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:21.379604 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:21.379895 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:21.379899 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:21.379900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:21.380148 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:21.380286 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:21.380519 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 14:33:21.380650 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 14:33:21.395920 30905 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:33:21.396935 30905 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:33:21.404258 30905 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a4f1e0194fd4628a481c4978e339150: Generated new TSK 0
I20260808 14:33:21.405006 30905 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:33:21.414968 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 30819
2026-08-08T14:33:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1487 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:33:21 dist-test-slave-w206 krb5kdc[30927](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:33:21 dist-test-slave-w206 krb5kdc[30927](info): set up 2 sockets
Aug 08 14:33:21 dist-test-slave-w206 krb5kdc[30927](info): commencing operation
krb5kdc: starting...
W20260808 14:33:22.484169 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:33:22 dist-test-slave-w206 krb5kdc[30927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:33:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:33:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:33:22.614380 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:33173
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:34779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:33173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:33:22.728309 30941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:33:22.728609 30941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:33:22.728696 30941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:33:22.732802 30941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:33:22.732892 30941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:33:22.732928 30941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:33:22.732985 30941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:33:22.733022 30941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:33:22.738749 30941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:34779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:33173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:33173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:33:22.739966 30941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:33:22.741076 30941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:33:22.749799 30949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:22.749861 30946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:22.749799 30947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:22.749966 30941 server_base.cc:1034] running on GCE node
I20260808 14:33:22.750808 30941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:33:22.751973 30941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:33:22.753173 30941 hybrid_clock.cc:648] HybridClock initialized: now 1786199602753124 us; error 62 us; skew 500 ppm
Aug 08 14:33:22 dist-test-slave-w206 krb5kdc[30927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199602, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:33:22.762362 30941 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:33:22.763518 30941 webserver.cc:460] Webserver started at http://127.7.125.126:45595/ using document root <none> and password file <none>
I20260808 14:33:22.764168 30941 fs_manager.cc:359] Metadata directory not provided
I20260808 14:33:22.764271 30941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:33:22.764545 30941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:33:22.766844 30941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "9e457e1c82164a898a9a854c7cb2dce0"
format_stamp: "Formatted at 2026-08-08 14:33:22 on dist-test-slave-w206"
server_key: "92af2fb7efa5e893bfea9db914b64f4e"
server_key_iv: "46df2096b621a12bff7c7da59772696a"
server_key_version: "encryptionkey@0"
I20260808 14:33:22.767822 30941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "9e457e1c82164a898a9a854c7cb2dce0"
format_stamp: "Formatted at 2026-08-08 14:33:22 on dist-test-slave-w206"
server_key: "92af2fb7efa5e893bfea9db914b64f4e"
server_key_iv: "46df2096b621a12bff7c7da59772696a"
server_key_version: "encryptionkey@0"
I20260808 14:33:22.771965 30941 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:33:22.774492 30956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:33:22.775840 30941 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:33:22.775990 30941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "9e457e1c82164a898a9a854c7cb2dce0"
format_stamp: "Formatted at 2026-08-08 14:33:22 on dist-test-slave-w206"
server_key: "92af2fb7efa5e893bfea9db914b64f4e"
server_key_iv: "46df2096b621a12bff7c7da59772696a"
server_key_version: "encryptionkey@0"
I20260808 14:33:22.776110 30941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:33:22.786437 30941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:33:22.789852 30941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:33:22.790081 30941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:33:22.797617 30941 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:33173
I20260808 14:33:22.797631 31011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:33173 every 8 connection(s)
I20260808 14:33:22.798636 30941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:33:22.801607 31012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:33:22.801652 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 30941
I20260808 14:33:22.801815 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:33:22.802143 7669 external_mini_cluster.cc:1428] Setting key b885059dc58fc2b995c0b7933e9c6564
I20260808 14:33:22.808816 31012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e457e1c82164a898a9a854c7cb2dce0: Bootstrap starting.
I20260808 14:33:22.811486 31012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e457e1c82164a898a9a854c7cb2dce0: Neither blocks nor log segments found. Creating new log.
I20260808 14:33:22.811635 31015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:22.803602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56373 (local address 127.7.125.126:33173)
0808 14:33:22.804040 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:22.804056 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:22.804093 (+ 37us) server_negotiation.cc:407] Connection header received
0808 14:33:22.804897 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:22.804915 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:22.805260 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:22.805541 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:22.805907 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:22.806904 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:22.808238 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:22.808725 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:22.808859 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:22.809763 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:22.809768 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:22.809799 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:22.810183 (+ 384us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:22.810239 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:22.810260 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:22.810349 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:22.810544 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 14:33:22.810810 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":150,"threads_started":1}
I20260808 14:33:22.812292 31012 log.cc:824] T 00000000000000000000000000000000 P 9e457e1c82164a898a9a854c7cb2dce0: Log is configured to *not* fsync() on all Append() calls
I20260808 14:33:22.814391 31012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e457e1c82164a898a9a854c7cb2dce0: No bootstrap required, opened a new log
I20260808 14:33:22.814577 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:33:22.819497 31012 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e457e1c82164a898a9a854c7cb2dce0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e457e1c82164a898a9a854c7cb2dce0" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33173 } }
I20260808 14:33:22.819820 31012 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e457e1c82164a898a9a854c7cb2dce0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:33:22.819876 31012 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e457e1c82164a898a9a854c7cb2dce0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e457e1c82164a898a9a854c7cb2dce0, State: Initialized, Role: FOLLOWER
I20260808 14:33:22.820426 31012 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e457e1c82164a898a9a854c7cb2dce0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e457e1c82164a898a9a854c7cb2dce0" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33173 } }
I20260808 14:33:22.820562 31012 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e457e1c82164a898a9a854c7cb2dce0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:33:22.820657 31012 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e457e1c82164a898a9a854c7cb2dce0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:33:22.820770 31012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e457e1c82164a898a9a854c7cb2dce0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:33:22.821628 31015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:22.817023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36145 (local address 127.7.125.126:33173)
0808 14:33:22.817163 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:22.817168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:22.817325 (+ 157us) server_negotiation.cc:407] Connection header received
0808 14:33:22.817421 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:22.817423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:22.817479 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:22.817572 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:22.817865 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:22.818700 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:22.819647 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:22.820010 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:22.820127 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:22.821213 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:22.821217 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:22.821220 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:22.821234 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:22.821264 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:22.821277 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:22.821353 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:22.821432 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 14:33:22.821494 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:33:22.822077 31012 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e457e1c82164a898a9a854c7cb2dce0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e457e1c82164a898a9a854c7cb2dce0" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33173 } }
I20260808 14:33:22.822469 31012 leader_election.cc:302] T 00000000000000000000000000000000 P 9e457e1c82164a898a9a854c7cb2dce0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e457e1c82164a898a9a854c7cb2dce0; no voters:
I20260808 14:33:22.822829 31012 leader_election.cc:288] T 00000000000000000000000000000000 P 9e457e1c82164a898a9a854c7cb2dce0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:33:22.822911 31022 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e457e1c82164a898a9a854c7cb2dce0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:33:22.823226 31022 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e457e1c82164a898a9a854c7cb2dce0 [term 1 LEADER]: Becoming Leader. State: Replica: 9e457e1c82164a898a9a854c7cb2dce0, State: Running, Role: LEADER
I20260808 14:33:22.823591 31022 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e457e1c82164a898a9a854c7cb2dce0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e457e1c82164a898a9a854c7cb2dce0" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33173 } }
I20260808 14:33:22.824198 31012 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e457e1c82164a898a9a854c7cb2dce0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:33:22.825340 31024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e457e1c82164a898a9a854c7cb2dce0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e457e1c82164a898a9a854c7cb2dce0. Latest consensus state: current_term: 1 leader_uuid: "9e457e1c82164a898a9a854c7cb2dce0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e457e1c82164a898a9a854c7cb2dce0" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33173 } } }
I20260808 14:33:22.825474 31024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e457e1c82164a898a9a854c7cb2dce0 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:22.825889 31031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:33:22.825834 31023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e457e1c82164a898a9a854c7cb2dce0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e457e1c82164a898a9a854c7cb2dce0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e457e1c82164a898a9a854c7cb2dce0" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 33173 } } }
I20260808 14:33:22.826325 31023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e457e1c82164a898a9a854c7cb2dce0 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:22.829180 31031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:33:22.835664 31031 catalog_manager.cc:1347] Generated new cluster ID: e835e056bb9c4a6aa1a90d05372ef9ff
I20260808 14:33:22.835736 31031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:33:22.847159 31031 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:33:22.848523 31031 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:33:22.856597 31031 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e457e1c82164a898a9a854c7cb2dce0: Generated new TSK 0
I20260808 14:33:22.857483 31031 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:33:22.861912 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 30941
2026-08-08T14:33:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1445 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:33:22 dist-test-slave-w206 krb5kdc[31044](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:33:22 dist-test-slave-w206 krb5kdc[31044](info): set up 2 sockets
Aug 08 14:33:22 dist-test-slave-w206 krb5kdc[31044](info): commencing operation
krb5kdc: starting...
W20260808 14:33:23.977177 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:33:24 dist-test-slave-w206 krb5kdc[31044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:33:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:33:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:33:24.110366 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:34863
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:34863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:33:24.227659 31058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:33:24.228003 31058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:33:24.228102 31058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:33:24.232332 31058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:33:24.232434 31058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:33:24.232481 31058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:33:24.232542 31058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:33:24.232589 31058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:33:24.239338 31058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:34863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:34863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:33:24.240864 31058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:33:24.242250 31058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:33:24.251058 31066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:24.251107 31064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:24.251093 31063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:24.251377 31058 server_base.cc:1034] running on GCE node
I20260808 14:33:24.251941 31058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:33:24.253137 31058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:33:24.254333 31058 hybrid_clock.cc:648] HybridClock initialized: now 1786199604254326 us; error 32 us; skew 500 ppm
Aug 08 14:33:24 dist-test-slave-w206 krb5kdc[31044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199604, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:33:24.257171 31058 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:33:24.258090 31058 webserver.cc:460] Webserver started at http://127.7.125.126:38295/ using document root <none> and password file <none>
I20260808 14:33:24.258688 31058 fs_manager.cc:359] Metadata directory not provided
I20260808 14:33:24.258795 31058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:33:24.259080 31058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:33:24.261199 31058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "10414129b32e4bbc8469b593b0c5f291"
format_stamp: "Formatted at 2026-08-08 14:33:24 on dist-test-slave-w206"
server_key: "37e387e57695ee97e7b1bb24163712ef"
server_key_iv: "ad40c0301b9185bdea5682243042fbef"
server_key_version: "encryptionkey@0"
I20260808 14:33:24.261782 31058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "10414129b32e4bbc8469b593b0c5f291"
format_stamp: "Formatted at 2026-08-08 14:33:24 on dist-test-slave-w206"
server_key: "37e387e57695ee97e7b1bb24163712ef"
server_key_iv: "ad40c0301b9185bdea5682243042fbef"
server_key_version: "encryptionkey@0"
I20260808 14:33:24.265614 31058 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 14:33:24.268113 31074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:33:24.269488 31058 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:33:24.269735 31058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "10414129b32e4bbc8469b593b0c5f291"
format_stamp: "Formatted at 2026-08-08 14:33:24 on dist-test-slave-w206"
server_key: "37e387e57695ee97e7b1bb24163712ef"
server_key_iv: "ad40c0301b9185bdea5682243042fbef"
server_key_version: "encryptionkey@0"
I20260808 14:33:24.269857 31058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:33:24.274576 31058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:33:24.277477 31058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:33:24.277670 31058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:33:24.285070 31058 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:34863
I20260808 14:33:24.285101 31126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:34863 every 8 connection(s)
I20260808 14:33:24.286082 31058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:33:24.287864 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 31058
I20260808 14:33:24.288012 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:33:24.288314 7669 external_mini_cluster.cc:1428] Setting key 1dc9adcf5cbfc4bdcd9b910e3c1d38c5
I20260808 14:33:24.289386 31127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:33:24.293787 31130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:24.289763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47671 (local address 127.7.125.126:34863)
0808 14:33:24.290276 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:24.290291 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:24.290350 (+ 59us) server_negotiation.cc:407] Connection header received
0808 14:33:24.291073 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:24.291091 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:24.291379 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:24.291707 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:24.292546 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:24.292551 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:24.292563 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:24.292573 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:24.292627 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:24.292646 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:24.292801 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:24.292874 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 14:33:24.293040 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":170,"threads_started":1}
I20260808 14:33:24.296217 31127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10414129b32e4bbc8469b593b0c5f291: Bootstrap starting.
I20260808 14:33:24.296911 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:33:24.299124 31127 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10414129b32e4bbc8469b593b0c5f291: Neither blocks nor log segments found. Creating new log.
I20260808 14:33:24.300108 31127 log.cc:824] T 00000000000000000000000000000000 P 10414129b32e4bbc8469b593b0c5f291: Log is configured to *not* fsync() on all Append() calls
I20260808 14:33:24.301184 31130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:24.299337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46837 (local address 127.7.125.126:34863)
0808 14:33:24.299481 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:24.299485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:24.299674 (+ 189us) server_negotiation.cc:407] Connection header received
0808 14:33:24.299805 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:24.299807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:24.299848 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:24.299956 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:24.300734 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:24.300737 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:24.300739 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:33:24.300749 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:24.300774 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:33:24.300783 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:24.300860 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:24.301013 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 14:33:24.301049 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 14:33:24.302428 31127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10414129b32e4bbc8469b593b0c5f291: No bootstrap required, opened a new log
I20260808 14:33:24.305934 31127 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10414129b32e4bbc8469b593b0c5f291 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10414129b32e4bbc8469b593b0c5f291" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 34863 } }
I20260808 14:33:24.306159 31127 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10414129b32e4bbc8469b593b0c5f291 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:33:24.306203 31127 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10414129b32e4bbc8469b593b0c5f291 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10414129b32e4bbc8469b593b0c5f291, State: Initialized, Role: FOLLOWER
I20260808 14:33:24.306581 31127 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10414129b32e4bbc8469b593b0c5f291 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10414129b32e4bbc8469b593b0c5f291" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 34863 } }
I20260808 14:33:24.306752 31127 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10414129b32e4bbc8469b593b0c5f291 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:33:24.306835 31127 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10414129b32e4bbc8469b593b0c5f291 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:33:24.306942 31127 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10414129b32e4bbc8469b593b0c5f291 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:33:24.307935 31127 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10414129b32e4bbc8469b593b0c5f291 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10414129b32e4bbc8469b593b0c5f291" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 34863 } }
I20260808 14:33:24.308288 31127 leader_election.cc:302] T 00000000000000000000000000000000 P 10414129b32e4bbc8469b593b0c5f291 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10414129b32e4bbc8469b593b0c5f291; no voters:
I20260808 14:33:24.308609 31127 leader_election.cc:288] T 00000000000000000000000000000000 P 10414129b32e4bbc8469b593b0c5f291 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:33:24.308743 31137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10414129b32e4bbc8469b593b0c5f291 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:33:24.309000 31137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10414129b32e4bbc8469b593b0c5f291 [term 1 LEADER]: Becoming Leader. State: Replica: 10414129b32e4bbc8469b593b0c5f291, State: Running, Role: LEADER
I20260808 14:33:24.309377 31137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10414129b32e4bbc8469b593b0c5f291 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10414129b32e4bbc8469b593b0c5f291" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 34863 } }
I20260808 14:33:24.309896 31127 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10414129b32e4bbc8469b593b0c5f291 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:33:24.311043 31138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10414129b32e4bbc8469b593b0c5f291 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10414129b32e4bbc8469b593b0c5f291" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10414129b32e4bbc8469b593b0c5f291" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 34863 } } }
I20260808 14:33:24.311188 31138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10414129b32e4bbc8469b593b0c5f291 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:24.311610 31147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:33:24.311573 31139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10414129b32e4bbc8469b593b0c5f291 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10414129b32e4bbc8469b593b0c5f291. Latest consensus state: current_term: 1 leader_uuid: "10414129b32e4bbc8469b593b0c5f291" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10414129b32e4bbc8469b593b0c5f291" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 34863 } } }
I20260808 14:33:24.311759 31139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10414129b32e4bbc8469b593b0c5f291 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:24.314877 31147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:33:24.321372 31147 catalog_manager.cc:1347] Generated new cluster ID: 6bae29433d41497ca8647e4e0ebe8c27
I20260808 14:33:24.321486 31147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:33:24.333663 31147 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:33:24.333997 31147 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:33:24.338923 31147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10414129b32e4bbc8469b593b0c5f291: Generated new TSK 0
I20260808 14:33:24.339643 31147 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:33:24.344215 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 31058
2026-08-08T14:33:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1482 ms)
[----------] 18 tests from AuthTokenIssuingTest (26514 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:33:24 dist-test-slave-w206 krb5kdc[31159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:33:24 dist-test-slave-w206 krb5kdc[31159](info): set up 2 sockets
Aug 08 14:33:24 dist-test-slave-w206 krb5kdc[31159](info): commencing operation
krb5kdc: starting...
W20260808 14:33:25.423532 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:33:25 dist-test-slave-w206 krb5kdc[31159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:33:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:33:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:33:25.559094 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:39915
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:46069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:39915,127.7.125.125:39033,127.7.125.124:33331
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:33:25.672760 31173 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 14:33:25.673137 31173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:33:25.673190 31173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:33:25.673246 31173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:33:25.677241 31173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:33:25.677333 31173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:33:25.677373 31173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:33:25.677410 31173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:33:25.677445 31173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:33:25.682699 31173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:46069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:39915,127.7.125.125:39033,127.7.125.124:33331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:39915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:33:25.683882 31173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:33:25.685000 31173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:33:25.693691 31182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:25.693696 31179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:25.693877 31173 server_base.cc:1034] running on GCE node
W20260808 14:33:25.693687 31180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:25.694517 31173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:33:25.695786 31173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:33:25.696954 31173 hybrid_clock.cc:648] HybridClock initialized: now 1786199605696888 us; error 87 us; skew 500 ppm
Aug 08 14:33:25 dist-test-slave-w206 krb5kdc[31159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199605, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:33:25.699889 31173 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:33:25.700809 31173 webserver.cc:460] Webserver started at http://127.7.125.126:46385/ using document root <none> and password file <none>
I20260808 14:33:25.701406 31173 fs_manager.cc:359] Metadata directory not provided
I20260808 14:33:25.701473 31173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:33:25.701736 31173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:33:25.703897 31173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "85b96564c46444348ab228fe24943e97"
format_stamp: "Formatted at 2026-08-08 14:33:25 on dist-test-slave-w206"
server_key: "f4b5fc69aedd904c4ff5bed5a9a29ea7"
server_key_iv: "2a78b765cf3722c2c38b94a1b9b990a6"
server_key_version: "encryptionkey@0"
I20260808 14:33:25.704474 31173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "85b96564c46444348ab228fe24943e97"
format_stamp: "Formatted at 2026-08-08 14:33:25 on dist-test-slave-w206"
server_key: "f4b5fc69aedd904c4ff5bed5a9a29ea7"
server_key_iv: "2a78b765cf3722c2c38b94a1b9b990a6"
server_key_version: "encryptionkey@0"
I20260808 14:33:25.708341 31173 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:33:25.710788 31190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:33:25.712057 31173 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:33:25.712203 31173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "85b96564c46444348ab228fe24943e97"
format_stamp: "Formatted at 2026-08-08 14:33:25 on dist-test-slave-w206"
server_key: "f4b5fc69aedd904c4ff5bed5a9a29ea7"
server_key_iv: "2a78b765cf3722c2c38b94a1b9b990a6"
server_key_version: "encryptionkey@0"
I20260808 14:33:25.712344 31173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:33:25.725591 31173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:33:25.728647 31173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:33:25.728857 31173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:33:25.735978 31173 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:39915
I20260808 14:33:25.736306 31244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:39915 every 8 connection(s)
I20260808 14:33:25.737113 31173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:33:25.740378 31245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:33:25.745021 31245 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39915 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:33:25.746568 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 31173
I20260808 14:33:25.746721 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:33:25.747141 7669 external_mini_cluster.cc:1428] Setting key de9fd64384f7ba6665df94ff8388b48d
Aug 08 14:33:25 dist-test-slave-w206 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199605, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.125.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.125@KRBTEST.COM" created.
I20260808 14:33:25.761891 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:25.747549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:39915 (local address 127.0.0.1:38978)
0808 14:33:25.748052 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:25.748107 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 14:33:25.749121 (+ 1014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:33:25.749802 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:33:25.749807 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:33:25.750290 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:33:25.750524 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:25.750567 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:25.752358 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:25.752363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:25.753516 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:25.753526 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:25.754613 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:25.754615 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:25.754687 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:25.755354 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:33:25.755383 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:33:25.757049 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:33:25.759087 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:25.759098 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:25.759108 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:25.759333 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:25.759680 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:25.759686 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:25.759690 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:25.759851 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:25.760551 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:33:25.760555 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:33:25.760664 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 14:33:25.760798 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 14:33:25.761052 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":135,"threads_started":1}
I20260808 14:33:25.761896 31251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:25.748429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38978 (local address 127.7.125.126:39915)
0808 14:33:25.748811 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:25.748822 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:25.748864 (+ 42us) server_negotiation.cc:407] Connection header received
0808 14:33:25.749432 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:25.749438 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:25.749622 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:25.749806 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:25.750719 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:25.752210 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:25.753650 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:25.754279 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:25.754453 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:25.757194 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:25.757200 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:25.757213 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:25.757236 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:25.758951 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:25.759457 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:25.759463 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:25.759465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:25.759577 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:25.759963 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:25.759969 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:25.759971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:25.760362 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:25.760519 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:25.760956 (+ 437us) server_negotiation.cc:299] Negotiation successful
0808 14:33:25.761137 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":79,"threads_started":1}
I20260808 14:33:25.764480 31245 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:33:25.765377 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:25.765056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:39033 (local address 127.0.0.1:48730)
0808 14:33:25.765187 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:25.765297 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.125:39033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 14:33:25.765616 31192 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.125.125:39033: connect: Connection refused (error 111)
I20260808 14:33:25.766459 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:25.766245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:39033 (local address 127.0.0.1:48732)
0808 14:33:25.766352 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:25.766399 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.125:39033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 14:33:25.766620 31245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.125:39033: Network error: Client connection negotiation failed: client connection to 127.7.125.125:39033: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.125.keytab.
Entry for principal kudu/127.7.125.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.125.keytab.
I20260808 14:33:25.777962 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.125:39033
--webserver_interface=127.7.125.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.125.keytab
--principal=kudu/127.7.125.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:46069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:39915,127.7.125.125:39033,127.7.125.124:33331
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:33:25.815858 31245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } attempt: 1
I20260808 14:33:25.816931 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:25.816535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:39033 (local address 127.0.0.1:48736)
0808 14:33:25.816748 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:25.816847 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.125:39033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 14:33:25.817740 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:25.817517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:39033 (local address 127.0.0.1:48746)
0808 14:33:25.817628 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:25.817684 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.125:39033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 14:33:25.817875 31245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.125:39033: Network error: Client connection negotiation failed: client connection to 127.7.125.125:39033: connect: Connection refused (error 111)
I20260808 14:33:25.886058 31245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } attempt: 2
I20260808 14:33:25.887108 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:25.886681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:39033 (local address 127.0.0.1:48752)
0808 14:33:25.886923 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:25.887017 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.125:39033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260808 14:33:25.887941 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:25.887701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:39033 (local address 127.0.0.1:48756)
0808 14:33:25.887825 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:25.887880 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.125:39033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 14:33:25.888100 31245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.125:39033: Network error: Client connection negotiation failed: client connection to 127.7.125.125:39033: connect: Connection refused (error 111)
W20260808 14:33:25.891894 31255 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 14:33:25.892143 31255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:33:25.892167 31255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:33:25.892212 31255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:33:25.896184 31255 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:33:25.896232 31255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:33:25.896291 31255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:33:25.896343 31255 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:33:25.896368 31255 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:33:25.901698 31255 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:46069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:39915,127.7.125.125:39033,127.7.125.124:33331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.125.keytab
--principal=kudu/127.7.125.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.125:39033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.125.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:33:25.902669 31255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:33:25.903712 31255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:33:25.911875 31263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:25.911871 31261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:25.911901 31260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:25.912315 31255 server_base.cc:1034] running on GCE node
I20260808 14:33:25.912873 31255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:33:25.913992 31255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:33:25.915148 31255 hybrid_clock.cc:648] HybridClock initialized: now 1786199605915128 us; error 30 us; skew 500 ppm
Aug 08 14:33:25 dist-test-slave-w206 krb5kdc[31159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199605, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:33:25.918079 31255 init.cc:377] Logged in from keytab as kudu/127.7.125.125@KRBTEST.COM (short username kudu)
I20260808 14:33:25.919080 31255 webserver.cc:460] Webserver started at http://127.7.125.125:44017/ using document root <none> and password file <none>
I20260808 14:33:25.919647 31255 fs_manager.cc:359] Metadata directory not provided
I20260808 14:33:25.919709 31255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:33:25.919961 31255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:33:25.922024 31255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-1/data/instance:
uuid: "e6bc7850dbd54cb99742909c8289880b"
format_stamp: "Formatted at 2026-08-08 14:33:25 on dist-test-slave-w206"
server_key: "3bf174bfbfb740d89aa3c8d13dae5081"
server_key_iv: "5ec26be7be2ece1a0a02b7b4074d095c"
server_key_version: "encryptionkey@0"
I20260808 14:33:25.922621 31255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-1/wal/instance:
uuid: "e6bc7850dbd54cb99742909c8289880b"
format_stamp: "Formatted at 2026-08-08 14:33:25 on dist-test-slave-w206"
server_key: "3bf174bfbfb740d89aa3c8d13dae5081"
server_key_iv: "5ec26be7be2ece1a0a02b7b4074d095c"
server_key_version: "encryptionkey@0"
I20260808 14:33:25.926468 31255 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:33:25.928912 31270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:33:25.930148 31255 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 14:33:25.930312 31255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-1/wal
uuid: "e6bc7850dbd54cb99742909c8289880b"
format_stamp: "Formatted at 2026-08-08 14:33:25 on dist-test-slave-w206"
server_key: "3bf174bfbfb740d89aa3c8d13dae5081"
server_key_iv: "5ec26be7be2ece1a0a02b7b4074d095c"
server_key_version: "encryptionkey@0"
I20260808 14:33:25.930428 31255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:33:25.940225 31255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:33:25.943440 31255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:33:25.943691 31255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:33:25.951222 31255 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.125:39033
I20260808 14:33:25.951514 31328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.125:39033 every 8 connection(s)
I20260808 14:33:25.952303 31255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-1/data/info.pb
I20260808 14:33:25.955103 31329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:33:25.955320 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 31255
I20260808 14:33:25.955470 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-1/wal/instance
I20260808 14:33:25.955789 7669 external_mini_cluster.cc:1428] Setting key 11db5e95959d6af2b089e2fb17847aab
I20260808 14:33:25.959626 31329 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39915 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.125.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.124@KRBTEST.COM" created.
Aug 08 14:33:25 dist-test-slave-w206 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199605, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.125@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:33:25.974691 31251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:25.961583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38988 (local address 127.7.125.126:39915)
0808 14:33:25.961724 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:25.961728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:25.962885 (+ 1157us) server_negotiation.cc:407] Connection header received
0808 14:33:25.963901 (+ 1016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:25.963905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:25.963970 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:25.964075 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:25.965257 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:25.965859 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:25.967168 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:25.967503 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:25.967620 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:25.970342 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:25.970346 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:25.970348 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:25.970375 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:25.972292 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:25.972810 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:25.972817 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:25.972822 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:25.972928 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:25.973344 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:25.973350 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:25.973353 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:25.973613 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:25.973728 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:25.974274 (+ 546us) server_negotiation.cc:299] Negotiation successful
0808 14:33:25.974439 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 14:33:25.975323 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:25.962138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:39915 (local address 127.0.0.1:38988)
0808 14:33:25.962716 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:25.962758 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 14:33:25.963668 (+ 910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:33:25.964250 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:33:25.964257 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:33:25.964618 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:33:25.965103 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:25.965116 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:25.966004 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:25.966007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:25.967035 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:25.967045 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:25.967664 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:25.967668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:25.967870 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:25.968518 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:33:25.968535 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:33:25.970105 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:33:25.972414 (+ 2309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:25.972419 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:25.972432 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:25.972695 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:25.973053 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:25.973058 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:25.973061 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:25.973236 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:25.973729 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:33:25.973734 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:33:25.973989 (+ 255us) client_negotiation.cc:769] Sending connection context
0808 14:33:25.974146 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 14:33:25.974405 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":177,"threads_started":1}
I20260808 14:33:25.976790 31329 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:33:25.979269 31245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } attempt: 3
Aug 08 14:33:25 dist-test-slave-w206 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199605, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.125@KRBTEST.COM for kudu/127.7.125.125@KRBTEST.COM
Aug 08 14:33:25 dist-test-slave-w206 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199605, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for kudu/127.7.125.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.124.keytab.
Entry for principal kudu/127.7.125.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.124.keytab.
I20260808 14:33:25.986946 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.124:33331
--webserver_interface=127.7.125.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.124.keytab
--principal=kudu/127.7.125.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:46069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:39915,127.7.125.125:39033,127.7.125.124:33331
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:33:25.988706 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:25.977410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:39033 (local address 127.0.0.1:48772)
0808 14:33:25.977536 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:25.977547 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 14:33:25.977610 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:33:25.978356 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:33:25.978359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:33:25.978494 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:33:25.978643 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:25.978650 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:25.980053 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:25.980056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:25.980773 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:25.980783 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:25.981729 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:25.981731 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:25.981797 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:25.982341 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:33:25.982352 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:33:25.983888 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:33:25.986342 (+ 2454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:25.986346 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:25.986349 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:25.986596 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:25.986943 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:25.986946 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:25.986948 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:25.987082 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:25.987680 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:33:25.987685 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:33:25.987815 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 14:33:25.987963 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 14:33:25.988445 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 14:33:25.988888 31335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:25.977562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48772 (local address 127.7.125.125:39033)
0808 14:33:25.977953 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:25.977962 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:25.977982 (+ 20us) server_negotiation.cc:407] Connection header received
0808 14:33:25.978018 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:25.978032 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:25.978180 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:25.978295 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:25.978785 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:25.979900 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:25.980920 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:25.981467 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:25.981575 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:25.984064 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:25.984074 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:25.984086 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:25.984107 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:25.986220 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:25.986736 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:25.986742 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:25.986743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:25.986835 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:25.987198 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:25.987203 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:25.987207 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:25.987543 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:25.987691 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:25.988091 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 14:33:25.988550 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":270,"server-negotiator.queue_time_us":304,"thread_start_us":177,"threads_started":1}
I20260808 14:33:25.990996 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:25.979728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:39033 (local address 127.0.0.1:48786)
0808 14:33:25.979865 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:25.979880 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:33:25.979983 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:33:25.980355 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:33:25.980357 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:33:25.980494 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:33:25.980587 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:25.980592 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:25.981597 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:25.981601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:25.982811 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:25.982823 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:25.983398 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:25.983402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:25.983560 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:25.984069 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:33:25.984082 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:33:25.985499 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:33:25.989276 (+ 3777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:25.989283 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:25.989287 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:25.989606 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:25.989964 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:25.989968 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:25.989970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:25.990067 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:25.990438 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:33:25.990441 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:33:25.990549 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 14:33:25.990657 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 14:33:25.990807 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 14:33:25.991010 31336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:25.979783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48786 (local address 127.7.125.125:39033)
0808 14:33:25.980128 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:25.980137 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:25.980148 (+ 11us) server_negotiation.cc:407] Connection header received
0808 14:33:25.980177 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:25.980180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:25.980219 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:25.980303 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:25.980715 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:25.981462 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:25.982951 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:25.983268 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:25.983395 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:25.985719 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:25.985723 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:25.985726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:25.985751 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:25.989144 (+ 3393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:25.989724 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:25.989729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:25.989731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:25.989821 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:25.990165 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:25.990168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:25.990170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:25.990332 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:25.990446 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:25.990708 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 14:33:25.990810 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":131,"threads_started":1}
I20260808 14:33:25.991264 31329 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:33:25.991950 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:25.991715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:33331 (local address 127.0.0.1:47890)
0808 14:33:25.991830 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:25.991900 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:33331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 14:33:25.992064 31245 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } has no permanent_uuid. Determining permanent_uuid...
W20260808 14:33:25.992274 31273 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.125.124:33331: connect: Connection refused (error 111)
I20260808 14:33:25.992727 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:25.992509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:33331 (local address 127.0.0.1:47906)
0808 14:33:25.992614 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:25.992662 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:33331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 14:33:25.993458 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:25.993200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:33331 (local address 127.0.0.1:47908)
0808 14:33:25.993354 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:25.993395 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:33331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260808 14:33:25.993628 31245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.124:33331: Network error: Client connection negotiation failed: client connection to 127.7.125.124:33331: connect: Connection refused (error 111)
I20260808 14:33:25.995631 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:25.995435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:33331 (local address 127.0.0.1:47916)
0808 14:33:25.995542 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:25.995584 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:33331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 14:33:25.995855 31329 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.124:33331: Network error: Client connection negotiation failed: client connection to 127.7.125.124:33331: connect: Connection refused (error 111)
I20260808 14:33:26.024799 31245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } attempt: 1
I20260808 14:33:26.025842 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.025417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:33331 (local address 127.0.0.1:47932)
0808 14:33:26.025635 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:26.025741 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:33331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260808 14:33:26.026739 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.026464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:33331 (local address 127.0.0.1:47942)
0808 14:33:26.026596 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:26.026663 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:33331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 14:33:26.026903 31245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.124:33331: Network error: Client connection negotiation failed: client connection to 127.7.125.124:33331: connect: Connection refused (error 111)
I20260808 14:33:26.045101 31329 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } attempt: 1
I20260808 14:33:26.045969 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.045661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:33331 (local address 127.0.0.1:47958)
0808 14:33:26.045806 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:26.045891 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:33331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 14:33:26.046764 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.046550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:33331 (local address 127.0.0.1:47974)
0808 14:33:26.046656 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:26.046704 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:33331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 14:33:26.046917 31329 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.124:33331: Network error: Client connection negotiation failed: client connection to 127.7.125.124:33331: connect: Connection refused (error 111)
I20260808 14:33:26.102087 31245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } attempt: 2
I20260808 14:33:26.102959 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.102619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:33331 (local address 127.0.0.1:47990)
0808 14:33:26.102796 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:26.102868 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:33331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 14:33:26.103688 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.103466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:33331 (local address 127.0.0.1:48000)
0808 14:33:26.103575 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:26.103627 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:33331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 14:33:26.103829 31245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.124:33331: Network error: Client connection negotiation failed: client connection to 127.7.125.124:33331: connect: Connection refused (error 111)
W20260808 14:33:26.108867 31337 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 14:33:26.109151 31337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:33:26.109222 31337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:33:26.109310 31337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:33:26.113390 31337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:33:26.113446 31337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:33:26.113466 31337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:33:26.113482 31337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:33:26.113497 31337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:33:26.115084 31329 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } attempt: 2
I20260808 14:33:26.115839 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.115569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:33331 (local address 127.0.0.1:48014)
0808 14:33:26.115701 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:26.115768 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:33331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 14:33:26.116565 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.116359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:33331 (local address 127.0.0.1:48018)
0808 14:33:26.116464 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:26.116516 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:33331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 14:33:26.116688 31329 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.124:33331: Network error: Client connection negotiation failed: client connection to 127.7.125.124:33331: connect: Connection refused (error 111)
I20260808 14:33:26.119447 31337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:46069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:39915,127.7.125.125:39033,127.7.125.124:33331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.124.keytab
--principal=kudu/127.7.125.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.124:33331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.125.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:33:26.120522 31337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:33:26.121539 31337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:33:26.130451 31347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:26.130436 31345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:26.130440 31343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:26.130496 31337 server_base.cc:1034] running on GCE node
I20260808 14:33:26.131294 31337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:33:26.132401 31337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:33:26.133594 31337 hybrid_clock.cc:648] HybridClock initialized: now 1786199606133556 us; error 47 us; skew 500 ppm
Aug 08 14:33:26 dist-test-slave-w206 krb5kdc[31159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199606, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:33:26.136758 31337 init.cc:377] Logged in from keytab as kudu/127.7.125.124@KRBTEST.COM (short username kudu)
I20260808 14:33:26.137746 31337 webserver.cc:460] Webserver started at http://127.7.125.124:35049/ using document root <none> and password file <none>
I20260808 14:33:26.138365 31337 fs_manager.cc:359] Metadata directory not provided
I20260808 14:33:26.138473 31337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:33:26.138746 31337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:33:26.141013 31337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-2/data/instance:
uuid: "65a8f37838bc4db1992f0fee88ec628e"
format_stamp: "Formatted at 2026-08-08 14:33:26 on dist-test-slave-w206"
server_key: "a4cb5d7b9692a43039dd6cafb3d3100e"
server_key_iv: "51871f1a2cb095aecd5ee44c91e390a3"
server_key_version: "encryptionkey@0"
I20260808 14:33:26.141649 31337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-2/wal/instance:
uuid: "65a8f37838bc4db1992f0fee88ec628e"
format_stamp: "Formatted at 2026-08-08 14:33:26 on dist-test-slave-w206"
server_key: "a4cb5d7b9692a43039dd6cafb3d3100e"
server_key_iv: "51871f1a2cb095aecd5ee44c91e390a3"
server_key_version: "encryptionkey@0"
I20260808 14:33:26.145927 31337 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 14:33:26.148618 31355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:33:26.149888 31337 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:33:26.150046 31337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-2/wal
uuid: "65a8f37838bc4db1992f0fee88ec628e"
format_stamp: "Formatted at 2026-08-08 14:33:26 on dist-test-slave-w206"
server_key: "a4cb5d7b9692a43039dd6cafb3d3100e"
server_key_iv: "51871f1a2cb095aecd5ee44c91e390a3"
server_key_version: "encryptionkey@0"
I20260808 14:33:26.150173 31337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:33:26.178853 31337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:33:26.182132 31337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:33:26.182359 31337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:33:26.189776 31337 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.124:33331
I20260808 14:33:26.190076 31412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.124:33331 every 8 connection(s)
I20260808 14:33:26.190881 31337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-2/data/info.pb
I20260808 14:33:26.193806 31413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:33:26.196647 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 31337
I20260808 14:33:26.196792 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199316863421-7669-0/minicluster-data/master-2/wal/instance
I20260808 14:33:26.197141 7669 external_mini_cluster.cc:1428] Setting key 8ee17751bcb88e1a13f7468599f93a24
I20260808 14:33:26.198329 31413 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39915 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:33:26.203004 31245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } attempt: 3
Aug 08 14:33:26 dist-test-slave-w206 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.124@KRBTEST.COM
Aug 08 14:33:26 dist-test-slave-w206 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199606, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.124@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:33:26.207834 31329 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } attempt: 3
Aug 08 14:33:26 dist-test-slave-w206 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199605, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for kudu/127.7.125.124@KRBTEST.COM
I20260808 14:33:26.213228 31251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.200468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39004 (local address 127.7.125.126:39915)
0808 14:33:26.200629 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.200635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.200908 (+ 273us) server_negotiation.cc:407] Connection header received
0808 14:33:26.201150 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.201153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.201197 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.201287 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.202146 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.202641 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.204054 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.204366 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.204462 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:26.208799 (+ 4337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:26.208803 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:26.208807 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:26.208837 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:26.210781 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:26.211564 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:26.211570 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:26.211572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:26.211669 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:26.212055 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:26.212060 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:26.212063 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:26.212369 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:26.212762 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:26.212877 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 14:33:26.213008 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 14:33:26.214474 31419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.200384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:39915 (local address 127.0.0.1:39004)
0808 14:33:26.200767 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:26.200828 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 14:33:26.200995 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:33:26.201348 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:33:26.201353 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:33:26.201719 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:33:26.202001 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:26.202012 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.202754 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.202758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:26.203928 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:26.203934 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.204662 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.204666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:26.204892 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:26.206175 (+ 1283us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:33:26.206194 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:33:26.208574 (+ 2380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:33:26.211039 (+ 2465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:26.211052 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:26.211065 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:26.211437 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:26.211781 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:26.211785 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:26.211787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:26.211943 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:26.212501 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:33:26.212508 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:33:26.212635 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 14:33:26.212806 (+ 171us) client_negotiation.cc:240] Negotiation successful
0808 14:33:26.213415 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":96,"thread_start_us":109,"threads_started":1}
Aug 08 14:33:26 dist-test-slave-w206 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199605, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.125@KRBTEST.COM for kudu/127.7.125.124@KRBTEST.COM
I20260808 14:33:26.214474 31417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.198730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48030 (local address 127.7.125.124:33331)
0808 14:33:26.199203 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.199226 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.199273 (+ 47us) server_negotiation.cc:407] Connection header received
0808 14:33:26.200188 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.200207 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.200656 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.201009 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.201957 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.202837 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.204040 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.204510 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.204826 (+ 316us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:26.207762 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:26.207773 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:26.207783 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:26.207824 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:26.211117 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:26.211635 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:26.211641 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:26.211643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:26.211781 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:26.212122 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:26.212125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:26.212127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:26.212616 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:26.213654 (+ 1038us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:26.213760 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 14:33:26.213900 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":299,"spinlock_wait_cycles":6528,"thread_start_us":165,"threads_started":1}
I20260808 14:33:26.214944 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.203410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:33331 (local address 127.0.0.1:48036)
0808 14:33:26.203573 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:26.203585 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:33:26.203652 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:33:26.204060 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:33:26.204063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:33:26.204254 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:33:26.204363 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:26.204368 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.205452 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.205459 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:26.205957 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:26.205964 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.206589 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.206593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:26.206701 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:26.207219 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:33:26.207240 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:33:26.208792 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:33:26.212365 (+ 3573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:26.212369 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:26.212371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:26.212645 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:26.213757 (+ 1112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:26.213761 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:26.213763 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:26.213859 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:26.214389 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:33:26.214393 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:33:26.214505 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 14:33:26.214600 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 14:33:26.214763 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":1}
I20260808 14:33:26.215955 31420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.203530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48036 (local address 127.7.125.124:33331)
0808 14:33:26.203842 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.203850 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.203864 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:33:26.203895 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.203897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.203934 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.204478 (+ 544us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.204570 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.205343 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.206077 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.206448 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.206554 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:26.208940 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:26.208944 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:26.208947 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:26.208973 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:26.212237 (+ 3264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:26.213514 (+ 1277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:26.213520 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:26.213523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:26.213615 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:26.214004 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:26.214009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:26.214014 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:26.214249 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:26.215489 (+ 1240us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:26.215581 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 14:33:26.215704 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":247,"thread_start_us":164,"threads_started":1}
I20260808 14:33:26.217372 31413 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:33:26.220181 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.208327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:33331 (local address 127.0.0.1:48048)
0808 14:33:26.208621 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:26.208638 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:33:26.208746 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:33:26.209671 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:33:26.209677 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:33:26.209876 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:33:26.210005 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:26.210017 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.211236 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.211240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:26.212135 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:26.212150 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.212821 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.212826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:26.212926 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:26.213413 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:33:26.213432 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:33:26.215024 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:33:26.218238 (+ 3214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:26.218245 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:26.218247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:26.218540 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:26.219038 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:26.219042 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:26.219044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:26.219156 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:26.219661 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:33:26.219664 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:33:26.219768 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 14:33:26.219854 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 14:33:26.220019 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208}
I20260808 14:33:26.220533 31245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97: Bootstrap starting.
I20260808 14:33:26.221095 31421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.208973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48048 (local address 127.7.125.124:33331)
0808 14:33:26.209377 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.209389 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.209405 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:33:26.209448 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.209451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.209503 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.210132 (+ 629us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.210233 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.211101 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.212316 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.212666 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.213161 (+ 495us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:26.215272 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:26.215276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:26.215280 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:26.215305 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:26.217641 (+ 2336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:26.218796 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:26.218799 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:26.218801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:26.218893 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:26.219277 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:26.219280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:26.219282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:26.219529 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:26.220732 (+ 1203us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:26.220811 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 14:33:26.220924 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":139,"threads_started":1}
I20260808 14:33:26.223815 31245 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97: Neither blocks nor log segments found. Creating new log.
I20260808 14:33:26.224843 31245 log.cc:824] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:33:26 dist-test-slave-w206 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199606, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.124@KRBTEST.COM for kudu/127.7.125.125@KRBTEST.COM
Aug 08 14:33:26 dist-test-slave-w206 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:33:26.225832 31329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6bc7850dbd54cb99742909c8289880b: Bootstrap starting.
I20260808 14:33:26.228319 31245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97: No bootstrap required, opened a new log
I20260808 14:33:26.228986 31329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6bc7850dbd54cb99742909c8289880b: Neither blocks nor log segments found. Creating new log.
I20260808 14:33:26.229919 31329 log.cc:824] T 00000000000000000000000000000000 P e6bc7850dbd54cb99742909c8289880b: Log is configured to *not* fsync() on all Append() calls
I20260808 14:33:26.230743 31251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.219151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39008 (local address 127.7.125.126:39915)
0808 14:33:26.219283 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.219288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.219304 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:33:26.219349 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.219355 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.219416 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.219498 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.220517 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.221265 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.222440 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.222739 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.222842 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:26.226211 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:26.226215 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:26.226218 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:26.226243 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:26.228096 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:26.228665 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:26.228669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:26.228671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:26.228760 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:26.229121 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:26.229125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:26.229127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:26.229356 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:26.229529 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:26.230474 (+ 945us) server_negotiation.cc:299] Negotiation successful
0808 14:33:26.230600 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:33:26.230801 31419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.217922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:39033 (local address 127.0.0.1:48796)
0808 14:33:26.219308 (+ 1386us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:26.219325 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:33:26.219460 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:33:26.219898 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:33:26.219901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:33:26.220122 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:33:26.220265 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:26.220274 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.221426 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.221429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:26.222284 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:26.222296 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.223111 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.223115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:26.223222 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:26.223780 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:33:26.223796 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:33:26.226483 (+ 2687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:33:26.228864 (+ 2381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:26.228871 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:26.228876 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:26.229151 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:26.229537 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:26.229540 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:26.229542 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:26.229643 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:26.230206 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:33:26.230210 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:33:26.230327 (+ 117us) client_negotiation.cc:769] Sending connection context
0808 14:33:26.230453 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 14:33:26.230622 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1314}
I20260808 14:33:26.230799 31336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.218219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48796 (local address 127.7.125.125:39033)
0808 14:33:26.218702 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.218709 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.219386 (+ 677us) server_negotiation.cc:407] Connection header received
0808 14:33:26.219582 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.219587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.219640 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.219709 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.220424 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.221263 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.222445 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.222826 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.222909 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:26.226651 (+ 3742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:26.226656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:26.226659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:26.226699 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:26.228718 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:26.229300 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:26.229305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:26.229310 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:26.229406 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:26.229780 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:26.229786 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:26.229789 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:26.230047 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:26.230152 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:26.230504 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 14:33:26.230626 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405}
I20260808 14:33:26.232872 31245 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b96564c46444348ab228fe24943e97" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39915 } } peers { permanent_uuid: "e6bc7850dbd54cb99742909c8289880b" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } } peers { permanent_uuid: "65a8f37838bc4db1992f0fee88ec628e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } }
I20260808 14:33:26.233156 31245 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85b96564c46444348ab228fe24943e97, State: Initialized, Role: FOLLOWER
I20260808 14:33:26.233518 31245 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b96564c46444348ab228fe24943e97" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39915 } } peers { permanent_uuid: "e6bc7850dbd54cb99742909c8289880b" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } } peers { permanent_uuid: "65a8f37838bc4db1992f0fee88ec628e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } }
I20260808 14:33:26.235019 31245 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:33:26.235366 31424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b96564c46444348ab228fe24943e97" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39915 } } peers { permanent_uuid: "e6bc7850dbd54cb99742909c8289880b" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } } peers { permanent_uuid: "65a8f37838bc4db1992f0fee88ec628e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } } }
I20260808 14:33:26.235531 31424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:33:26.235862 31413 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 14:33:26 dist-test-slave-w206 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.125@KRBTEST.COM
Aug 08 14:33:26 dist-test-slave-w206 krb5kdc[31159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199606, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.124@KRBTEST.COM for kudu/127.7.125.124@KRBTEST.COM
I20260808 14:33:26.240567 31173 master_runner.cc:186] Error getting master registration for 127.7.125.125:39033: OK, instance_id { permanent_uuid: "e6bc7850dbd54cb99742909c8289880b" instance_seqno: 1786199605949102 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 14:33:26.244552 31173 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 14:33:26.244827 31336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.232734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48810 (local address 127.7.125.125:39033)
0808 14:33:26.232866 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.232872 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.232889 (+ 17us) server_negotiation.cc:407] Connection header received
0808 14:33:26.232979 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.232984 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.233036 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.233138 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.234046 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.235021 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.236304 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.236664 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.236788 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:26.240089 (+ 3301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:26.240093 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:26.240096 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:26.240122 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:26.242141 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:26.242712 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:26.242716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:26.242717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:26.242807 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:26.243219 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:26.243226 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:26.243230 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:26.243479 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:26.243647 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:26.244307 (+ 660us) server_negotiation.cc:299] Negotiation successful
0808 14:33:26.244667 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 14:33:26.245072 31329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6bc7850dbd54cb99742909c8289880b: No bootstrap required, opened a new log
I20260808 14:33:26.248545 31419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.237180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:33331 (local address 127.0.0.1:48054)
0808 14:33:26.237394 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:26.237410 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:33:26.237535 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:33:26.238055 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:33:26.238058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:33:26.238369 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:33:26.238542 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:26.238556 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.239759 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.239762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:26.240524 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:26.240536 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.241252 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.241256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:26.241331 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:26.241757 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:33:26.241775 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:33:26.243387 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:33:26.245562 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:26.245565 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:26.245567 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:26.245850 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:26.246169 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:26.246173 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:26.246177 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:26.246272 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:26.247978 (+ 1706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:33:26.247985 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:33:26.248086 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 14:33:26.248207 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 14:33:26.248376 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143}
I20260808 14:33:26.248713 31421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.236804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48054 (local address 127.7.125.124:33331)
0808 14:33:26.237122 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.237128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.237469 (+ 341us) server_negotiation.cc:407] Connection header received
0808 14:33:26.237643 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.237646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.237710 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.237806 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.238696 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.239616 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.240657 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.241025 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.241104 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:26.243541 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:26.243545 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:26.243548 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:26.243574 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:26.245281 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:26.245962 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:26.245968 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:26.245971 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:26.246065 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:26.246373 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:26.246380 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:26.246381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:26.247752 (+ 1371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:26.247876 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:26.248298 (+ 422us) server_negotiation.cc:299] Negotiation successful
0808 14:33:26.248496 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":80}
I20260808 14:33:26.249330 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:33:26.249485 31329 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6bc7850dbd54cb99742909c8289880b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b96564c46444348ab228fe24943e97" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39915 } } peers { permanent_uuid: "e6bc7850dbd54cb99742909c8289880b" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } } peers { permanent_uuid: "65a8f37838bc4db1992f0fee88ec628e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } }
I20260808 14:33:26.249676 31329 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6bc7850dbd54cb99742909c8289880b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6bc7850dbd54cb99742909c8289880b, State: Initialized, Role: FOLLOWER
I20260808 14:33:26.249939 31329 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6bc7850dbd54cb99742909c8289880b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b96564c46444348ab228fe24943e97" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39915 } } peers { permanent_uuid: "e6bc7850dbd54cb99742909c8289880b" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } } peers { permanent_uuid: "65a8f37838bc4db1992f0fee88ec628e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } }
I20260808 14:33:26.250361 31442 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6bc7850dbd54cb99742909c8289880b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b96564c46444348ab228fe24943e97" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39915 } } peers { permanent_uuid: "e6bc7850dbd54cb99742909c8289880b" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } } peers { permanent_uuid: "65a8f37838bc4db1992f0fee88ec628e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } } }
I20260808 14:33:26.250476 31442 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6bc7850dbd54cb99742909c8289880b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:33:26.250787 31218 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "85b96564c46444348ab228fe24943e97"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39008
I20260808 14:33:26.251215 31218 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 14:33:26.251298 31329 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6bc7850dbd54cb99742909c8289880b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:33:26.251343 31218 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:33:26.252185 31413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65a8f37838bc4db1992f0fee88ec628e: Bootstrap starting.
I20260808 14:33:26.252753 31218 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b96564c46444348ab228fe24943e97" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39915 } } peers { permanent_uuid: "e6bc7850dbd54cb99742909c8289880b" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } } peers { permanent_uuid: "65a8f37838bc4db1992f0fee88ec628e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } }
W20260808 14:33:26.253199 31438 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:33:26.253261 31438 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:33:26.253793 31218 leader_election.cc:288] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97 [CANDIDATE]: Term 1 election: Requested vote from peers e6bc7850dbd54cb99742909c8289880b (127.7.125.125:39033), 65a8f37838bc4db1992f0fee88ec628e (127.7.125.124:33331)
I20260808 14:33:26.254794 31302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "85b96564c46444348ab228fe24943e97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e6bc7850dbd54cb99742909c8289880b"
I20260808 14:33:26.255036 31302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6bc7850dbd54cb99742909c8289880b [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:33:26.255280 31413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65a8f37838bc4db1992f0fee88ec628e: Neither blocks nor log segments found. Creating new log.
I20260808 14:33:26.256184 31413 log.cc:824] T 00000000000000000000000000000000 P 65a8f37838bc4db1992f0fee88ec628e: Log is configured to *not* fsync() on all Append() calls
I20260808 14:33:26.256505 31302 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e6bc7850dbd54cb99742909c8289880b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85b96564c46444348ab228fe24943e97 in term 1.
I20260808 14:33:26.257157 31192 leader_election.cc:302] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85b96564c46444348ab228fe24943e97, e6bc7850dbd54cb99742909c8289880b; no voters:
I20260808 14:33:26.258442 31424 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:33:26.258782 31424 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97 [term 1 LEADER]: Becoming Leader. State: Replica: 85b96564c46444348ab228fe24943e97, State: Running, Role: LEADER
I20260808 14:33:26.259234 31424 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b96564c46444348ab228fe24943e97" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39915 } } peers { permanent_uuid: "e6bc7850dbd54cb99742909c8289880b" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } } peers { permanent_uuid: "65a8f37838bc4db1992f0fee88ec628e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } }
W20260808 14:33:26.260016 31453 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e6bc7850dbd54cb99742909c8289880b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:33:26.260094 31453 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:33:26.262145 31413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65a8f37838bc4db1992f0fee88ec628e: No bootstrap required, opened a new log
I20260808 14:33:26.266950 31413 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65a8f37838bc4db1992f0fee88ec628e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b96564c46444348ab228fe24943e97" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39915 } } peers { permanent_uuid: "e6bc7850dbd54cb99742909c8289880b" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } } peers { permanent_uuid: "65a8f37838bc4db1992f0fee88ec628e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } }
I20260808 14:33:26.267261 31413 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65a8f37838bc4db1992f0fee88ec628e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65a8f37838bc4db1992f0fee88ec628e, State: Initialized, Role: FOLLOWER
I20260808 14:33:26.267601 31413 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65a8f37838bc4db1992f0fee88ec628e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b96564c46444348ab228fe24943e97" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39915 } } peers { permanent_uuid: "e6bc7850dbd54cb99742909c8289880b" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } } peers { permanent_uuid: "65a8f37838bc4db1992f0fee88ec628e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } }
I20260808 14:33:26.268996 31413 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65a8f37838bc4db1992f0fee88ec628e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:33:26.271061 31385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "85b96564c46444348ab228fe24943e97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "65a8f37838bc4db1992f0fee88ec628e"
I20260808 14:33:26.273211 31385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65a8f37838bc4db1992f0fee88ec628e [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:33:26.274626 31385 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 65a8f37838bc4db1992f0fee88ec628e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85b96564c46444348ab228fe24943e97 in term 1.
W20260808 14:33:26.277992 31477 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 65a8f37838bc4db1992f0fee88ec628e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:33:26.278067 31477 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:33:26.269471 31255 master_runner.cc:186] Error getting master registration for 127.7.125.124:33331: OK, instance_id { permanent_uuid: "65a8f37838bc4db1992f0fee88ec628e" instance_seqno: 1786199606187734 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 14:33:26.278694 31255 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 14:33:26.279119 31251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.260733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39014 (local address 127.7.125.126:39915)
0808 14:33:26.260857 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.260862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.262688 (+ 1826us) server_negotiation.cc:407] Connection header received
0808 14:33:26.262906 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.262910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.262964 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.263041 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.264220 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.265009 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.266359 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.266721 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.266809 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:26.274686 (+ 7877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:26.274691 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:26.274694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:26.274723 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:26.276739 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:26.277318 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:26.277322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:26.277324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:26.277425 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:26.277824 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:26.277830 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:26.277833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:26.278089 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:26.278207 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:26.278708 (+ 501us) server_negotiation.cc:299] Negotiation successful
0808 14:33:26.278854 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 14:33:26.280932 31466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65a8f37838bc4db1992f0fee88ec628e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b96564c46444348ab228fe24943e97" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39915 } } peers { permanent_uuid: "e6bc7850dbd54cb99742909c8289880b" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } } peers { permanent_uuid: "65a8f37838bc4db1992f0fee88ec628e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } } }
I20260808 14:33:26.281059 31466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65a8f37838bc4db1992f0fee88ec628e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:33:26.282565 31458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85b96564c46444348ab228fe24943e97. Latest consensus state: current_term: 1 leader_uuid: "85b96564c46444348ab228fe24943e97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b96564c46444348ab228fe24943e97" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39915 } } peers { permanent_uuid: "e6bc7850dbd54cb99742909c8289880b" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } } peers { permanent_uuid: "65a8f37838bc4db1992f0fee88ec628e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } } }
I20260808 14:33:26.282744 31458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:26.285948 31336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.261587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48820 (local address 127.7.125.125:39033)
0808 14:33:26.261732 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.261737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.268024 (+ 6287us) server_negotiation.cc:407] Connection header received
0808 14:33:26.268221 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.268224 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.268276 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.268331 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.269531 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.270268 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.271601 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.271973 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.272055 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:26.281492 (+ 9437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:26.281496 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:26.281499 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:26.281531 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:26.283508 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:26.284194 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:26.284200 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:26.284202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:26.284308 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:26.284716 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:26.284722 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:26.284725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:26.285007 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:26.285170 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:26.285552 (+ 382us) server_negotiation.cc:299] Negotiation successful
0808 14:33:26.285711 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 14:33:26.287477 31479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:33:26.288493 31479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:33:26.289772 31421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:26.261976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48058 (local address 127.7.125.124:33331)
0808 14:33:26.262129 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.262134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.278002 (+ 15868us) server_negotiation.cc:407] Connection header received
0808 14:33:26.278045 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.278048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.278097 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.278191 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.279850 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.280993 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.283208 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:26.283589 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:26.283671 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:26.285529 (+ 1858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:26.285534 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:26.285536 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:26.285566 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:26.287477 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:26.288058 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:26.288062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:26.288064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:26.288260 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:26.288656 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:26.288661 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:26.288664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:26.288916 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:26.289042 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:26.289400 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 14:33:26.289564 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 14:33:26.300470 31302 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e6bc7850dbd54cb99742909c8289880b [term 1 FOLLOWER]: Refusing update from remote peer 85b96564c46444348ab228fe24943e97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:33:26.301232 31385 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 65a8f37838bc4db1992f0fee88ec628e [term 1 FOLLOWER]: Refusing update from remote peer 85b96564c46444348ab228fe24943e97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:33:26.302021 31424 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65a8f37838bc4db1992f0fee88ec628e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:33:26.302188 31458 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6bc7850dbd54cb99742909c8289880b" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:33:26.307050 31466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65a8f37838bc4db1992f0fee88ec628e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85b96564c46444348ab228fe24943e97. Latest consensus state: current_term: 1 leader_uuid: "85b96564c46444348ab228fe24943e97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b96564c46444348ab228fe24943e97" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39915 } } peers { permanent_uuid: "e6bc7850dbd54cb99742909c8289880b" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } } peers { permanent_uuid: "65a8f37838bc4db1992f0fee88ec628e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } } }
I20260808 14:33:26.307250 31466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65a8f37838bc4db1992f0fee88ec628e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:33:26.307278 31442 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6bc7850dbd54cb99742909c8289880b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85b96564c46444348ab228fe24943e97. Latest consensus state: current_term: 1 leader_uuid: "85b96564c46444348ab228fe24943e97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b96564c46444348ab228fe24943e97" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39915 } } peers { permanent_uuid: "e6bc7850dbd54cb99742909c8289880b" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } } peers { permanent_uuid: "65a8f37838bc4db1992f0fee88ec628e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } } }
I20260808 14:33:26.307397 31442 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6bc7850dbd54cb99742909c8289880b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:33:26.311960 31458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "85b96564c46444348ab228fe24943e97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b96564c46444348ab228fe24943e97" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39915 } } peers { permanent_uuid: "e6bc7850dbd54cb99742909c8289880b" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } } peers { permanent_uuid: "65a8f37838bc4db1992f0fee88ec628e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } } }
I20260808 14:33:26.312131 31458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:26.312359 31442 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6bc7850dbd54cb99742909c8289880b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "85b96564c46444348ab228fe24943e97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b96564c46444348ab228fe24943e97" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39915 } } peers { permanent_uuid: "e6bc7850dbd54cb99742909c8289880b" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } } peers { permanent_uuid: "65a8f37838bc4db1992f0fee88ec628e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } } }
I20260808 14:33:26.312492 31442 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6bc7850dbd54cb99742909c8289880b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:33:26.312506 31480 mvcc.cc:205] Tried to move back new op lower bound from 7316273587398631424 to 7316273587251638272. Current Snapshot: MvccSnapshot[applied={T|T < 7316273587398631424}]
I20260808 14:33:26.315518 31482 mvcc.cc:205] Tried to move back new op lower bound from 7316273587398631424 to 7316273587251638272. Current Snapshot: MvccSnapshot[applied={T|T < 7316273587398631424}]
I20260808 14:33:26.326596 31424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "85b96564c46444348ab228fe24943e97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b96564c46444348ab228fe24943e97" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39915 } } peers { permanent_uuid: "e6bc7850dbd54cb99742909c8289880b" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } } peers { permanent_uuid: "65a8f37838bc4db1992f0fee88ec628e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } } }
I20260808 14:33:26.326773 31424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:26.328092 31466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65a8f37838bc4db1992f0fee88ec628e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "85b96564c46444348ab228fe24943e97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b96564c46444348ab228fe24943e97" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 39915 } } peers { permanent_uuid: "e6bc7850dbd54cb99742909c8289880b" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 39033 } } peers { permanent_uuid: "65a8f37838bc4db1992f0fee88ec628e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 33331 } } }
I20260808 14:33:26.328251 31466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65a8f37838bc4db1992f0fee88ec628e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:33:26.329489 31479 catalog_manager.cc:1347] Generated new cluster ID: a3015fd52dab4514bd363b0affdc6a0b
I20260808 14:33:26.329566 31479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:33:26.332618 31481 mvcc.cc:205] Tried to move back new op lower bound from 7316273587398631424 to 7316273587251638272. Current Snapshot: MvccSnapshot[applied={T|T < 7316273587398631424}]
I20260808 14:33:26.379494 31479 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:33:26.380776 31479 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:33:26.403611 31479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85b96564c46444348ab228fe24943e97: Generated new TSK 0
I20260808 14:33:26.404511 31479 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 14:33:26.441984 31336 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:26.439335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48834 (local address 127.7.125.125:39033)
0808 14:33:26.439505 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.439510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.440745 (+ 1235us) server_negotiation.cc:407] Connection header received
0808 14:33:26.440968 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.440972 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.441026 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.441085 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.441831 (+ 746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48834: BlockingRecv error: recv got EOF from 127.0.0.1:48834 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 14:33:26.443570 31421 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:26.440146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48060 (local address 127.7.125.124:33331)
0808 14:33:26.440293 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.440298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.442523 (+ 2225us) server_negotiation.cc:407] Connection header received
0808 14:33:26.442731 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.442737 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.442806 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.442867 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.443436 (+ 569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48060: BlockingRecv error: recv got EOF from 127.0.0.1:48060 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 14:33:26.445439 31251 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:26.444370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39024 (local address 127.7.125.126:39915)
0808 14:33:26.444490 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.444494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.444531 (+ 37us) server_negotiation.cc:407] Connection header received
0808 14:33:26.444664 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.444667 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.444716 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.444815 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.445323 (+ 508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39024: BlockingRecv error: recv got EOF from 127.0.0.1:39024 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 14:33:26.450248 31336 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:26.448242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48850 (local address 127.7.125.125:39033)
0808 14:33:26.448360 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.448364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.448659 (+ 295us) server_negotiation.cc:407] Connection header received
0808 14:33:26.448969 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.448973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.449117 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.449223 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.450130 (+ 907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48850: BlockingRecv error: recv got EOF from 127.0.0.1:48850 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260808 14:33:26.456241 31336 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:26.454592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48854 (local address 127.7.125.125:39033)
0808 14:33:26.454698 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.454715 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.455317 (+ 602us) server_negotiation.cc:407] Connection header received
0808 14:33:26.455503 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.455507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.455557 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.455610 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.456133 (+ 523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48854: BlockingRecv error: recv got EOF from 127.0.0.1:48854 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 14:33:26.462313 31336 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:26.460636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48866 (local address 127.7.125.125:39033)
0808 14:33:26.460755 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.460760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.461267 (+ 507us) server_negotiation.cc:407] Connection header received
0808 14:33:26.461462 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.461466 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.461519 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.461572 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.462202 (+ 630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48866: BlockingRecv error: recv got EOF from 127.0.0.1:48866 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 14:33:26.470129 31336 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:26.468805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48868 (local address 127.7.125.125:39033)
0808 14:33:26.468963 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.468967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.469218 (+ 251us) server_negotiation.cc:407] Connection header received
0808 14:33:26.469377 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.469380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.469436 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.469533 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.470010 (+ 477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48868: BlockingRecv error: recv got EOF from 127.0.0.1:48868 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260808 14:33:26.481863 31336 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:26.480411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48874 (local address 127.7.125.125:39033)
0808 14:33:26.480969 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.480974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.480987 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:33:26.481038 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.481041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.481088 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.481163 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.481730 (+ 567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48874: BlockingRecv error: recv got EOF from 127.0.0.1:48874 (error 108)
Metrics: {"server-negotiator.queue_time_us":474}
W20260808 14:33:26.501392 31336 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:26.500052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48884 (local address 127.7.125.125:39033)
0808 14:33:26.500189 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.500194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.500448 (+ 254us) server_negotiation.cc:407] Connection header received
0808 14:33:26.500624 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.500634 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.500684 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.500775 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.501253 (+ 478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48884: BlockingRecv error: recv got EOF from 127.0.0.1:48884 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 14:33:26.536962 31336 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:26.535737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48900 (local address 127.7.125.125:39033)
0808 14:33:26.535897 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.535901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.536038 (+ 137us) server_negotiation.cc:407] Connection header received
0808 14:33:26.536148 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.536150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.536199 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.536357 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.536850 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48900: BlockingRecv error: recv got EOF from 127.0.0.1:48900 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260808 14:33:26.604732 31336 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:26.603397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48916 (local address 127.7.125.125:39033)
0808 14:33:26.603559 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.603566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.603778 (+ 212us) server_negotiation.cc:407] Connection header received
0808 14:33:26.603976 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.603980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.604030 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.604129 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.604627 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48916: BlockingRecv error: recv got EOF from 127.0.0.1:48916 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260808 14:33:26.736308 31336 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:26.735024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48922 (local address 127.7.125.125:39033)
0808 14:33:26.735166 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.735169 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.735365 (+ 196us) server_negotiation.cc:407] Connection header received
0808 14:33:26.735474 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.735477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.735524 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.735609 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.736199 (+ 590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48922: BlockingRecv error: recv got EOF from 127.0.0.1:48922 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 14:33:26.995820 31336 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:26.994567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48934 (local address 127.7.125.125:39033)
0808 14:33:26.994720 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:26.994726 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:26.994817 (+ 91us) server_negotiation.cc:407] Connection header received
0808 14:33:26.995038 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:26.995041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:26.995095 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:26.995192 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:26.995709 (+ 517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48934: BlockingRecv error: recv got EOF from 127.0.0.1:48934 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 14:33:27.261415 31453 catalog_manager.cc:1259] Loaded cluster ID: a3015fd52dab4514bd363b0affdc6a0b
I20260808 14:33:27.261509 31453 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e6bc7850dbd54cb99742909c8289880b: loading cluster ID for follower catalog manager: success
I20260808 14:33:27.263299 31453 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e6bc7850dbd54cb99742909c8289880b: acquiring CA information for follower catalog manager: success
I20260808 14:33:27.264529 31453 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e6bc7850dbd54cb99742909c8289880b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 14:33:27.279417 31477 catalog_manager.cc:1259] Loaded cluster ID: a3015fd52dab4514bd363b0affdc6a0b
I20260808 14:33:27.279500 31477 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 65a8f37838bc4db1992f0fee88ec628e: loading cluster ID for follower catalog manager: success
I20260808 14:33:27.281179 31477 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 65a8f37838bc4db1992f0fee88ec628e: acquiring CA information for follower catalog manager: success
I20260808 14:33:27.282354 31477 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 65a8f37838bc4db1992f0fee88ec628e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 14:33:27.513922 31555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:27.510118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48936 (local address 127.7.125.125:39033)
0808 14:33:27.510358 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:27.510362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:27.510381 (+ 19us) server_negotiation.cc:407] Connection header received
0808 14:33:27.510524 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:27.510526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:27.510657 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:27.510752 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:33:27.511104 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:27.511641 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:27.512504 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:27.512720 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:27.512795 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:27.513088 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:33:27.513524 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:33:27.513647 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:27.513710 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 14:33:27.513760 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":97,"threads_started":1}
I20260808 14:33:27.520903 31562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:27.517380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48072 (local address 127.7.125.124:33331)
0808 14:33:27.517622 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:27.517626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:27.517641 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:33:27.517722 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:27.517724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:27.517837 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:27.517953 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:33:27.518117 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:27.518620 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:27.519488 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:27.519715 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:27.519793 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:27.520077 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:33:27.520500 (+ 423us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:33:27.520591 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:27.520642 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 14:33:27.520689 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":73,"threads_started":1}
I20260808 14:33:27.528213 31555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:27.523786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48950 (local address 127.7.125.125:39033)
0808 14:33:27.523942 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:27.523949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:27.524434 (+ 485us) server_negotiation.cc:407] Connection header received
0808 14:33:27.524607 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:27.524615 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:27.524691 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:27.524782 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:33:27.525205 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:27.525965 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:27.526922 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:27.527277 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:27.527388 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:27.527682 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:33:27.527772 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:33:27.527838 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:27.528018 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 14:33:27.528080 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 14:33:27.528667 31562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:27.524271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48082 (local address 127.7.125.124:33331)
0808 14:33:27.524389 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:27.524395 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:27.524665 (+ 270us) server_negotiation.cc:407] Connection header received
0808 14:33:27.524796 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:27.524799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:27.524846 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:27.524917 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:33:27.525244 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:27.525744 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:27.526982 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:27.527302 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:27.527417 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:27.527733 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:33:27.527801 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:33:27.527867 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:27.528053 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 14:33:27.528107 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 14:33:27.528970 31568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:27.523926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39032 (local address 127.7.125.126:39915)
0808 14:33:27.524363 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:27.524370 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:27.524391 (+ 21us) server_negotiation.cc:407] Connection header received
0808 14:33:27.524491 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:27.524494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:27.524674 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:27.524870 (+ 196us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:33:27.525098 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:27.525960 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:27.527151 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:27.527493 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:27.527604 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:27.527933 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:33:27.528301 (+ 368us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:33:27.528693 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:27.528734 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 14:33:27.528813 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":78,"threads_started":1}
I20260808 14:33:27.532027 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 31173
I20260808 14:33:27.540669 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 31255
I20260808 14:33:27.548296 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 31337
2026-08-08T14:33:27Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3205 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:33:27 dist-test-slave-w206 krb5kdc[31573](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:33:27 dist-test-slave-w206 krb5kdc[31573](info): set up 2 sockets
Aug 08 14:33:27 dist-test-slave-w206 krb5kdc[31573](info): commencing operation
krb5kdc: starting...
W20260808 14:33:28.928757 7669 mini_kdc.cc:121] Time spent starting KDC: real 1.357s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/test-user.keytab.
Aug 08 14:33:29 dist-test-slave-w206 krb5kdc[31573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:33:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:33:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.125.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
Entry for principal kudu/127.7.125.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab.
I20260808 14:33:29.066625 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:36093
--webserver_interface=127.7.125.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:36093,127.7.125.125:41177,127.7.125.124:38677
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:33:29.185899 31587 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 14:33:29.186262 31587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:33:29.186292 31587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:33:29.186399 31587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:33:29.190694 31587 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:33:29.190780 31587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:33:29.190800 31587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:33:29.190817 31587 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:33:29.190833 31587 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:33:29.196550 31587 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:36093,127.7.125.125:41177,127.7.125.124:38677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.126.keytab
--principal=kudu/127.7.125.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.126:36093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.125.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:33:29.197830 31587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:33:29.199007 31587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:33:29.208178 31592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:29.208173 31595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:29.208220 31593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:29.208436 31587 server_base.cc:1034] running on GCE node
I20260808 14:33:29.209061 31587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:33:29.210192 31587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:33:29.211388 31587 hybrid_clock.cc:648] HybridClock initialized: now 1786199609211366 us; error 48 us; skew 500 ppm
Aug 08 14:33:29 dist-test-slave-w206 krb5kdc[31573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199609, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:33:29.220242 31587 init.cc:377] Logged in from keytab as kudu/127.7.125.126@KRBTEST.COM (short username kudu)
I20260808 14:33:29.221276 31587 webserver.cc:460] Webserver started at http://127.7.125.126:33361/ using document root <none> and password file <none>
I20260808 14:33:29.221866 31587 fs_manager.cc:359] Metadata directory not provided
I20260808 14:33:29.221930 31587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:33:29.222106 31587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:33:29.224258 31587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-0/data/instance:
uuid: "5d110290f8484d5693c4a1db44492a46"
format_stamp: "Formatted at 2026-08-08 14:33:29 on dist-test-slave-w206"
server_key: "e34b691939c59a25086d6406500381fb"
server_key_iv: "677ef2e24cbed41abd1d4bfa42fbdff9"
server_key_version: "encryptionkey@0"
I20260808 14:33:29.224789 31587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-0/wal/instance:
uuid: "5d110290f8484d5693c4a1db44492a46"
format_stamp: "Formatted at 2026-08-08 14:33:29 on dist-test-slave-w206"
server_key: "e34b691939c59a25086d6406500381fb"
server_key_iv: "677ef2e24cbed41abd1d4bfa42fbdff9"
server_key_version: "encryptionkey@0"
I20260808 14:33:29.228480 31587 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:33:29.230798 31603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:33:29.232197 31587 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 14:33:29.232394 31587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-0/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-0/wal
uuid: "5d110290f8484d5693c4a1db44492a46"
format_stamp: "Formatted at 2026-08-08 14:33:29 on dist-test-slave-w206"
server_key: "e34b691939c59a25086d6406500381fb"
server_key_iv: "677ef2e24cbed41abd1d4bfa42fbdff9"
server_key_version: "encryptionkey@0"
I20260808 14:33:29.232503 31587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:33:29.243294 31587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:33:29.246238 31587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:33:29.246472 31587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:33:29.254259 31587 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.126:36093
I20260808 14:33:29.254575 31657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.126:36093 every 8 connection(s)
I20260808 14:33:29.255348 31587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-0/data/info.pb
I20260808 14:33:29.258070 31658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:33:29.261941 31658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36093 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:33:29.264170 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 31587
I20260808 14:33:29.264318 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-0/wal/instance
I20260808 14:33:29.264660 7669 external_mini_cluster.cc:1428] Setting key c961433313efb00f22474e2c7a29abd1
Aug 08 14:33:29 dist-test-slave-w206 krb5kdc[31573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199609, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.125.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.125@KRBTEST.COM" created.
I20260808 14:33:29.279578 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.264385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36093 (local address 127.0.0.1:42722)
0808 14:33:29.264961 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.265020 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 14:33:29.266490 (+ 1470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:33:29.267489 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:33:29.267493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:33:29.267857 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:33:29.268186 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:29.268200 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.269899 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.269901 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:29.270727 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:29.270736 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.271758 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.271761 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:29.271870 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.272508 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:33:29.272524 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:33:29.274257 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:33:29.276186 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:29.276194 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:29.276202 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:29.276496 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:29.276914 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:29.276916 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:29.276918 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:29.277091 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:29.277867 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:33:29.277874 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:33:29.278004 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 14:33:29.278176 (+ 172us) client_negotiation.cc:240] Negotiation successful
0808 14:33:29.278546 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":184,"thread_start_us":148,"threads_started":1}
I20260808 14:33:29.279578 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.264385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42722 (local address 127.7.125.126:36093)
0808 14:33:29.264950 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.264969 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.265128 (+ 159us) server_negotiation.cc:407] Connection header received
0808 14:33:29.266906 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.266914 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.267256 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.267462 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.268352 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.269753 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.270861 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.271596 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.271835 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.274430 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:29.274440 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:29.274455 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:29.274483 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:29.276086 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.276694 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.276698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.276700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.276817 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.277191 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.277194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.277196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.277723 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:29.277880 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:29.278237 (+ 357us) server_negotiation.cc:299] Negotiation successful
0808 14:33:29.278495 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":8,"server-negotiator.queue_time_us":442,"spinlock_wait_cycles":25216,"thread_start_us":170,"threads_started":1}
I20260808 14:33:29.282225 31658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:33:29.283087 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.282769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:41177 (local address 127.0.0.1:53394)
0808 14:33:29.282901 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.283015 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.125:41177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260808 14:33:29.283321 31604 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.125.125:41177: connect: Connection refused (error 111)
I20260808 14:33:29.284133 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.283923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:41177 (local address 127.0.0.1:53402)
0808 14:33:29.284031 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.284080 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.125:41177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 14:33:29.284296 31658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.125:41177: Network error: Client connection negotiation failed: client connection to 127.7.125.125:41177: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.125.keytab.
Entry for principal kudu/127.7.125.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.125.keytab.
I20260808 14:33:29.294642 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.125:41177
--webserver_interface=127.7.125.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.125.keytab
--principal=kudu/127.7.125.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:36093,127.7.125.125:41177,127.7.125.124:38677
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:33:29.333628 31658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } attempt: 1
I20260808 14:33:29.334627 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.334252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:41177 (local address 127.0.0.1:53416)
0808 14:33:29.334444 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.334525 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.125:41177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 14:33:29.335477 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.335239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:41177 (local address 127.0.0.1:53418)
0808 14:33:29.335355 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.335410 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.125:41177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 14:33:29.335661 31658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.125:41177: Network error: Client connection negotiation failed: client connection to 127.7.125.125:41177: connect: Connection refused (error 111)
I20260808 14:33:29.403884 31658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } attempt: 2
I20260808 14:33:29.404783 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.404419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:41177 (local address 127.0.0.1:53432)
0808 14:33:29.404607 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.404682 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.125:41177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 14:33:29.405606 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.405343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:41177 (local address 127.0.0.1:53448)
0808 14:33:29.405475 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.405540 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.125:41177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 14:33:29.405820 31658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.125:41177: Network error: Client connection negotiation failed: client connection to 127.7.125.125:41177: connect: Connection refused (error 111)
W20260808 14:33:29.410776 31665 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 14:33:29.411273 31665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:33:29.411336 31665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:33:29.411409 31665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:33:29.415604 31665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:33:29.415689 31665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:33:29.415725 31665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:33:29.415762 31665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:33:29.415791 31665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:33:29.421787 31665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:36093,127.7.125.125:41177,127.7.125.124:38677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.125.keytab
--principal=kudu/127.7.125.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.125:41177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.125.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:33:29.423089 31665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:33:29.424196 31665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:33:29.433605 31671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:29.433604 31670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:29.433604 31673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:29.433795 31665 server_base.cc:1034] running on GCE node
I20260808 14:33:29.434506 31665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:33:29.435750 31665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:33:29.437011 31665 hybrid_clock.cc:648] HybridClock initialized: now 1786199609436996 us; error 35 us; skew 500 ppm
Aug 08 14:33:29 dist-test-slave-w206 krb5kdc[31573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199609, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:33:29.440001 31665 init.cc:377] Logged in from keytab as kudu/127.7.125.125@KRBTEST.COM (short username kudu)
I20260808 14:33:29.440932 31665 webserver.cc:460] Webserver started at http://127.7.125.125:37523/ using document root <none> and password file <none>
I20260808 14:33:29.441515 31665 fs_manager.cc:359] Metadata directory not provided
I20260808 14:33:29.441610 31665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:33:29.441862 31665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:33:29.444043 31665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-1/data/instance:
uuid: "43caba4d8b904590a3ecffb1905e1e9e"
format_stamp: "Formatted at 2026-08-08 14:33:29 on dist-test-slave-w206"
server_key: "8866028d6ccccacecbd017afd9603c38"
server_key_iv: "52468a17f0d5137179279fa6922d174a"
server_key_version: "encryptionkey@0"
I20260808 14:33:29.444654 31665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-1/wal/instance:
uuid: "43caba4d8b904590a3ecffb1905e1e9e"
format_stamp: "Formatted at 2026-08-08 14:33:29 on dist-test-slave-w206"
server_key: "8866028d6ccccacecbd017afd9603c38"
server_key_iv: "52468a17f0d5137179279fa6922d174a"
server_key_version: "encryptionkey@0"
I20260808 14:33:29.448443 31665 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:33:29.450941 31680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:33:29.452311 31665 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 14:33:29.452484 31665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-1/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-1/wal
uuid: "43caba4d8b904590a3ecffb1905e1e9e"
format_stamp: "Formatted at 2026-08-08 14:33:29 on dist-test-slave-w206"
server_key: "8866028d6ccccacecbd017afd9603c38"
server_key_iv: "52468a17f0d5137179279fa6922d174a"
server_key_version: "encryptionkey@0"
I20260808 14:33:29.452615 31665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:33:29.465207 31665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:33:29.468189 31665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:33:29.468416 31665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:33:29.475705 31665 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.125:41177
I20260808 14:33:29.475994 31741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.125:41177 every 8 connection(s)
I20260808 14:33:29.476720 31665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-1/data/info.pb
I20260808 14:33:29.479521 31742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:33:29.482244 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 31665
I20260808 14:33:29.482400 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-1/wal/instance
I20260808 14:33:29.482692 7669 external_mini_cluster.cc:1428] Setting key a24c28a746e6e0e4e1fa3d85f34a1612
I20260808 14:33:29.483489 31742 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36093 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.125.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.125.124@KRBTEST.COM" created.
Aug 08 14:33:29 dist-test-slave-w206 krb5kdc[31573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199609, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.125@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:33:29.497020 31658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } attempt: 3
I20260808 14:33:29.500698 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.485366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42728 (local address 127.7.125.126:36093)
0808 14:33:29.485532 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.485537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.486350 (+ 813us) server_negotiation.cc:407] Connection header received
0808 14:33:29.487358 (+ 1008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.487361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.487438 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.487542 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.488696 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.489230 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.490397 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.490638 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.490752 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.496169 (+ 5417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:29.496172 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:29.496175 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:29.496203 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:29.498339 (+ 2136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.499014 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.499020 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.499023 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.499114 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.499474 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.499479 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.499481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.499670 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:29.499778 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:29.500397 (+ 619us) server_negotiation.cc:299] Negotiation successful
0808 14:33:29.500525 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 14:33:29.501633 31745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.485753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36093 (local address 127.0.0.1:42728)
0808 14:33:29.486195 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.486239 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 14:33:29.487142 (+ 903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:33:29.487714 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:33:29.487721 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:33:29.488085 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:33:29.488545 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:29.488557 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.489374 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.489383 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:29.490258 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:29.490267 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.490804 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.490808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:29.491042 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.491697 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:33:29.491714 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:33:29.495942 (+ 4228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:33:29.498464 (+ 2522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:29.498474 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:29.498492 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:29.498880 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:29.499218 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:29.499221 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:29.499223 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:29.499377 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:29.499790 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:33:29.499800 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:33:29.500071 (+ 271us) client_negotiation.cc:769] Sending connection context
0808 14:33:29.500285 (+ 214us) client_negotiation.cc:240] Negotiation successful
0808 14:33:29.500628 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":102,"threads_started":1}
I20260808 14:33:29.503784 31742 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 14:33:29 dist-test-slave-w206 krb5kdc[31573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199609, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for kudu/127.7.125.125@KRBTEST.COM
I20260808 14:33:29.508819 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.497478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:41177 (local address 127.0.0.1:53452)
0808 14:33:29.497611 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.497629 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:33:29.497713 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:33:29.498617 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:33:29.498620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:33:29.498792 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:33:29.498907 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:29.498912 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.500532 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.500537 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:29.501510 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:29.501525 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.502352 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.502356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:29.502472 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.503134 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:33:29.503148 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:33:29.504718 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:33:29.507010 (+ 2292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:29.507013 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:29.507016 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:29.507247 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:29.507560 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:29.507564 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:29.507566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:29.507662 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:29.508235 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:33:29.508239 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:33:29.508348 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 14:33:29.508463 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 14:33:29.508624 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 14:33:29.509371 31747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.497616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53452 (local address 127.7.125.125:41177)
0808 14:33:29.498145 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.498159 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.498189 (+ 30us) server_negotiation.cc:407] Connection header received
0808 14:33:29.498241 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.498249 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.498493 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.498642 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.499054 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.500366 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.501652 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.502188 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.502330 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.504855 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:29.504866 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:29.504883 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:29.504912 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:29.506851 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.507366 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.507371 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.507372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.507450 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.507790 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.507795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.507797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.508119 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:29.508260 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:29.508594 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 14:33:29.508963 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":406,"thread_start_us":194,"threads_started":1}
Aug 08 14:33:29 dist-test-slave-w206 krb5kdc[31573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199609, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.125@KRBTEST.COM for kudu/127.7.125.125@KRBTEST.COM
I20260808 14:33:29.511384 31658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:33:29.512267 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.511928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:38677 (local address 127.0.0.1:51048)
0808 14:33:29.512066 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.512116 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:38677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 14:33:29.513074 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.512778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:38677 (local address 127.0.0.1:51058)
0808 14:33:29.512947 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.513002 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:38677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260808 14:33:29.513233 31658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.124:38677: Network error: Client connection negotiation failed: client connection to 127.7.125.124:38677: connect: Connection refused (error 111)
I20260808 14:33:29.513645 31745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.504385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:41177 (local address 127.0.0.1:53462)
0808 14:33:29.504580 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.504598 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:33:29.504685 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:33:29.504986 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:33:29.504989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:33:29.505188 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:33:29.505328 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:29.505336 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.506368 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.506373 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:29.507178 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:29.507186 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.507758 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.507764 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:29.508020 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.508519 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:33:29.508532 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:33:29.509938 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:33:29.511945 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:29.511951 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:29.511955 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:29.512187 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:29.512470 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:29.512473 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:29.512475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:29.512563 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:29.513083 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:33:29.513085 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:33:29.513287 (+ 202us) client_negotiation.cc:769] Sending connection context
0808 14:33:29.513406 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 14:33:29.513535 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":78}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.125.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.124.keytab.
Entry for principal kudu/127.7.125.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.124.keytab.
I20260808 14:33:29.513904 31748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.504441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53462 (local address 127.7.125.125:41177)
0808 14:33:29.504760 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.504772 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.504786 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:33:29.504823 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.504827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.504871 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.504978 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.505453 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.506224 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.507286 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.507645 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.507745 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.510082 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:29.510085 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:29.510087 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:29.510107 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:29.511826 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.512267 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.512270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.512272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.512368 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.512699 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.512705 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.512710 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.512976 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:29.513080 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:29.513523 (+ 443us) server_negotiation.cc:299] Negotiation successful
0808 14:33:29.513678 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":252,"thread_start_us":104,"threads_started":1}
I20260808 14:33:29.514550 31742 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:33:29.515244 7669 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
/tmp/dist-test-taskngm_Z6/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.124:38677
--webserver_interface=127.7.125.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.124.keytab
--principal=kudu/127.7.125.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.125.84:38297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.125.126:36093,127.7.125.125:41177,127.7.125.124:38677
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:33:29.515393 31745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.515078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:38677 (local address 127.0.0.1:51068)
0808 14:33:29.515188 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.515269 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:38677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 14:33:29.515605 31686 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.125.124:38677: connect: Connection refused (error 111)
I20260808 14:33:29.516491 31745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.516283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:38677 (local address 127.0.0.1:51084)
0808 14:33:29.516390 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.516436 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:38677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 14:33:29.516659 31742 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.124:38677: Network error: Client connection negotiation failed: client connection to 127.7.125.124:38677: connect: Connection refused (error 111)
I20260808 14:33:29.544428 31658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } attempt: 1
I20260808 14:33:29.545426 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.545019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:38677 (local address 127.0.0.1:51090)
0808 14:33:29.545247 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.545329 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:38677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 14:33:29.546195 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.545966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:38677 (local address 127.0.0.1:51102)
0808 14:33:29.546078 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.546130 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:38677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 14:33:29.546388 31658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.124:38677: Network error: Client connection negotiation failed: client connection to 127.7.125.124:38677: connect: Connection refused (error 111)
I20260808 14:33:29.565899 31742 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } attempt: 1
I20260808 14:33:29.566819 31745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.566509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:38677 (local address 127.0.0.1:51114)
0808 14:33:29.566675 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.566738 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:38677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 14:33:29.567595 31745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.567368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:38677 (local address 127.0.0.1:51122)
0808 14:33:29.567490 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.567541 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:38677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 14:33:29.567727 31742 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.124:38677: Network error: Client connection negotiation failed: client connection to 127.7.125.124:38677: connect: Connection refused (error 111)
I20260808 14:33:29.621596 31658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } attempt: 2
I20260808 14:33:29.622510 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.622170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:38677 (local address 127.0.0.1:51134)
0808 14:33:29.622346 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.622419 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:38677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 14:33:29.623303 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.623085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:38677 (local address 127.0.0.1:51144)
0808 14:33:29.623191 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.623243 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:38677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 14:33:29.623443 31658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.124:38677: Network error: Client connection negotiation failed: client connection to 127.7.125.124:38677: connect: Connection refused (error 111)
W20260808 14:33:29.632546 31750 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 14:33:29.632817 31750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:33:29.632874 31750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:33:29.632963 31750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 14:33:29.635903 31742 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } attempt: 2
I20260808 14:33:29.636896 31745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.636592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:38677 (local address 127.0.0.1:51154)
0808 14:33:29.636733 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.636801 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:38677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 14:33:29.637233 31750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:33:29.637334 31750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:33:29.637380 31750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:33:29.637421 31750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:33:29.637501 31750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:33:29.637727 31745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.637496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:38677 (local address 127.0.0.1:51166)
0808 14:33:29.637610 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.637672 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.125.124:38677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 14:33:29.637945 31742 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.125.124:38677: Network error: Client connection negotiation failed: client connection to 127.7.125.124:38677: connect: Connection refused (error 111)
I20260808 14:33:29.643250 31750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.125.84:38297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.125.126:36093,127.7.125.125:41177,127.7.125.124:38677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/krb5kdc/kudu_127.7.125.124.keytab
--principal=kudu/127.7.125.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.125.124:38677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.125.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:33:29.644346 31750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:33:29.645381 31750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:33:29.654654 31750 server_base.cc:1034] running on GCE node
W20260808 14:33:29.654575 31759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:29.654590 31757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:33:29.654570 31756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:33:29.655515 31750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:33:29.656698 31750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:33:29.657893 31750 hybrid_clock.cc:648] HybridClock initialized: now 1786199609657882 us; error 40 us; skew 500 ppm
Aug 08 14:33:29 dist-test-slave-w206 krb5kdc[31573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199609, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:33:29.660771 31750 init.cc:377] Logged in from keytab as kudu/127.7.125.124@KRBTEST.COM (short username kudu)
I20260808 14:33:29.661716 31750 webserver.cc:460] Webserver started at http://127.7.125.124:35677/ using document root <none> and password file <none>
I20260808 14:33:29.662285 31750 fs_manager.cc:359] Metadata directory not provided
I20260808 14:33:29.662405 31750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:33:29.662638 31750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:33:29.664736 31750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-2/data/instance:
uuid: "0a9924acf27646ab9dc0c32f4c66db3e"
format_stamp: "Formatted at 2026-08-08 14:33:29 on dist-test-slave-w206"
server_key: "1a7afb4f9856076c8af2db8870d654c8"
server_key_iv: "7f5871c2638eb21ca8e0db03bf56d95f"
server_key_version: "encryptionkey@0"
I20260808 14:33:29.665289 31750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-2/wal/instance:
uuid: "0a9924acf27646ab9dc0c32f4c66db3e"
format_stamp: "Formatted at 2026-08-08 14:33:29 on dist-test-slave-w206"
server_key: "1a7afb4f9856076c8af2db8870d654c8"
server_key_iv: "7f5871c2638eb21ca8e0db03bf56d95f"
server_key_version: "encryptionkey@0"
I20260808 14:33:29.668979 31750 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 14:33:29.671368 31766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:33:29.672539 31750 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:33:29.672678 31750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-2/data,/tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-2/wal
uuid: "0a9924acf27646ab9dc0c32f4c66db3e"
format_stamp: "Formatted at 2026-08-08 14:33:29 on dist-test-slave-w206"
server_key: "1a7afb4f9856076c8af2db8870d654c8"
server_key_iv: "7f5871c2638eb21ca8e0db03bf56d95f"
server_key_version: "encryptionkey@0"
I20260808 14:33:29.672786 31750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:33:29.686672 31750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:33:29.690065 31750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:33:29.690280 31750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:33:29.697841 31750 rpc_server.cc:307] RPC server started. Bound to: 127.7.125.124:38677
I20260808 14:33:29.698168 31823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.125.124:38677 every 8 connection(s)
I20260808 14:33:29.698869 31750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-2/data/info.pb
I20260808 14:33:29.701732 31824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:33:29.703115 7669 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu as pid 31750
I20260808 14:33:29.703264 7669 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskngm_Z6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199316863421-7669-0/minicluster-data/master-2/wal/instance
I20260808 14:33:29.703595 7669 external_mini_cluster.cc:1428] Setting key 3050d165b27c2d46a0d8f1a25afc7ee2
I20260808 14:33:29.706243 31824 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36093 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 14:33:29 dist-test-slave-w206 krb5kdc[31573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.124@KRBTEST.COM
Aug 08 14:33:29 dist-test-slave-w206 krb5kdc[31573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199609, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.124@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
I20260808 14:33:29.717700 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.708364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42732 (local address 127.7.125.126:36093)
0808 14:33:29.708506 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.708511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.708817 (+ 306us) server_negotiation.cc:407] Connection header received
0808 14:33:29.708978 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.708981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.709035 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.709133 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.709794 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.710275 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.711219 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.711437 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.711549 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.714101 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:29.714104 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:29.714106 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:29.714130 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:29.715626 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.716139 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.716143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.716145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.716227 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.716576 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.716580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.716582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.716781 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:29.716901 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:29.717370 (+ 469us) server_negotiation.cc:299] Negotiation successful
0808 14:33:29.717526 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 14:33:29.718513 31827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.705014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51176 (local address 127.7.125.124:38677)
0808 14:33:29.705414 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.705432 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.705481 (+ 49us) server_negotiation.cc:407] Connection header received
0808 14:33:29.706137 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.706153 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.706491 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.706823 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.707784 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.709227 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.710123 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.710598 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.710765 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.713659 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:29.713665 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:29.713675 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:29.713697 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:29.715619 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.716096 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.716102 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.716104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.716217 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.716523 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.716526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.716528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.716913 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:29.717139 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:29.717320 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 14:33:29.717700 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":136,"server-negotiator.queue_time_us":263,"thread_start_us":134,"threads_started":1}
I20260808 14:33:29.718513 31828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.708273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.126:36093 (local address 127.0.0.1:42732)
0808 14:33:29.708685 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.708724 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 14:33:29.708876 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:33:29.709182 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:33:29.709187 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:33:29.709538 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:33:29.709654 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:29.709660 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.710406 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.710409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:29.711098 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:29.711108 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.711590 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.711594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:29.711699 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.712231 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:33:29.712246 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:33:29.713948 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:33:29.715768 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:29.715778 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:29.715788 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:29.716030 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:29.716327 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:29.716330 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:29.716332 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:29.716476 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:29.716902 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:33:29.716910 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:33:29.717049 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 14:33:29.717223 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 14:33:29.717609 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":4,"thread_start_us":121,"threads_started":1}
I20260808 14:33:29.720580 31824 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:33:29.722610 31658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } attempt: 3
Aug 08 14:33:29 dist-test-slave-w206 krb5kdc[31573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199609, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.124@KRBTEST.COM for kudu/127.7.125.125@KRBTEST.COM
Aug 08 14:33:29 dist-test-slave-w206 krb5kdc[31573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.126@KRBTEST.COM
Aug 08 14:33:29 dist-test-slave-w206 krb5kdc[31573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199609, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.126@KRBTEST.COM for kudu/127.7.125.124@KRBTEST.COM
I20260808 14:33:29.729110 31742 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } attempt: 3
I20260808 14:33:29.731951 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.722966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:38677 (local address 127.0.0.1:51188)
0808 14:33:29.723136 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.723149 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:33:29.723229 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:33:29.723447 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:33:29.723450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:33:29.723631 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:33:29.723721 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:29.723727 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.724653 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.724657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:29.725402 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:29.725415 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.725946 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.725950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:29.726029 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.726481 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:33:29.726495 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:33:29.728176 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:33:29.730047 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:29.730050 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:29.730052 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:29.730272 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:29.730736 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:29.730739 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:29.730741 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:29.730848 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:29.731366 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:33:29.731370 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:33:29.731481 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 14:33:29.731604 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 14:33:29.731757 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":2}
I20260808 14:33:29.732250 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.721388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42744 (local address 127.7.125.126:36093)
0808 14:33:29.721501 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.721505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.721518 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:33:29.721573 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.721579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.721622 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.721693 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.722406 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.723337 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.724634 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.724976 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.725086 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.727835 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:29.727841 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:29.727848 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:29.727877 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:29.729756 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.730315 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.730321 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.730326 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.730423 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.730746 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.730751 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.730753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.731064 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:29.731312 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:29.731941 (+ 629us) server_negotiation.cc:299] Negotiation successful
0808 14:33:29.732087 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":47}
I20260808 14:33:29.732405 31827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.723080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51188 (local address 127.7.125.124:38677)
0808 14:33:29.723183 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.723187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.723200 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:33:29.723290 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.723293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.723333 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.723416 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.723834 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.724530 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.725557 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.725823 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.725945 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.728303 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:29.728312 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:29.728316 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:29.728336 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:29.729943 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.730535 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.730541 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.730545 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.730637 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.731010 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.731014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.731016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.731231 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:29.731319 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:29.732159 (+ 840us) server_negotiation.cc:299] Negotiation successful
0808 14:33:29.732264 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 14:33:29.732729 31748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.721295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53464 (local address 127.7.125.125:41177)
0808 14:33:29.721433 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.721437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.721495 (+ 58us) server_negotiation.cc:407] Connection header received
0808 14:33:29.721612 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.721614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.721658 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.721756 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.722217 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.723007 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.724098 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.724456 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.724581 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.727480 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:29.727483 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:29.727486 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:29.727513 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:29.730286 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.730794 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.730797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.730799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.730889 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.731260 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.731265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.731269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.731484 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:29.731560 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:29.732428 (+ 868us) server_negotiation.cc:299] Negotiation successful
0808 14:33:29.732555 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 14:33:29.733410 31828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.721187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.125:41177 (local address 127.0.0.1:53464)
0808 14:33:29.721410 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.721424 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:33:29.721517 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:33:29.721798 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:33:29.721803 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:33:29.721966 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:33:29.722086 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:29.722091 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.723137 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.723142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:29.723976 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:29.723986 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.724587 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.724590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:29.724693 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.725417 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:33:29.725428 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:33:29.727337 (+ 1909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:33:29.730404 (+ 3067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:29.730407 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:29.730409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:29.730661 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:29.731051 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:29.731053 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:29.731055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:29.731148 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:29.731635 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:33:29.731638 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:33:29.731750 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 14:33:29.733104 (+ 1354us) client_negotiation.cc:240] Negotiation successful
0808 14:33:29.733259 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156}
I20260808 14:33:29.734089 31824 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 14:33:29 dist-test-slave-w206 krb5kdc[31573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199609, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.125@KRBTEST.COM for kudu/127.7.125.124@KRBTEST.COM
I20260808 14:33:29.736148 31658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46: Bootstrap starting.
I20260808 14:33:29.739310 31658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46: Neither blocks nor log segments found. Creating new log.
I20260808 14:33:29.740056 31745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.729539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:38677 (local address 127.0.0.1:51192)
0808 14:33:29.729638 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.729654 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:33:29.729746 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:33:29.730662 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:33:29.730668 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:33:29.730854 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:33:29.731009 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:29.731017 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.732543 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.732548 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:29.733291 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:29.733302 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.733961 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.733965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:29.734043 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.734500 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:33:29.734514 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:33:29.736101 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:33:29.738312 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:29.738316 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:29.738318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:29.738574 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:29.738888 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:29.738893 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:29.738895 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:29.739023 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:29.739520 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:33:29.739524 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:33:29.739634 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 14:33:29.739736 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 14:33:29.739876 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 14:33:29.740121 31829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.729921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51192 (local address 127.7.125.124:38677)
0808 14:33:29.730391 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.730407 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.730424 (+ 17us) server_negotiation.cc:407] Connection header received
0808 14:33:29.730463 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.730469 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.730522 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.731119 (+ 597us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.731193 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.732401 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.733443 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.733821 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.734251 (+ 430us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.736362 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:29.736365 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:29.736368 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:29.736400 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:29.738187 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.738698 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.738701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.738703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.738788 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.739146 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.739152 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.739156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.739377 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:29.739534 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:29.739834 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 14:33:29.739958 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":254,"threads_started":1}
I20260808 14:33:29.740290 31658 log.cc:824] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:33:29 dist-test-slave-w206 krb5kdc[31573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199609, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.125.124@KRBTEST.COM for kudu/127.7.125.124@KRBTEST.COM
I20260808 14:33:29.743166 31658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46: No bootstrap required, opened a new log
I20260808 14:33:29.743295 31742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43caba4d8b904590a3ecffb1905e1e9e: Bootstrap starting.
Aug 08 14:33:29 dist-test-slave-w206 krb5kdc[31573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.125.125@KRBTEST.COM
I20260808 14:33:29.745452 31827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.735392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51196 (local address 127.7.125.124:38677)
0808 14:33:29.735488 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.735492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.735963 (+ 471us) server_negotiation.cc:407] Connection header received
0808 14:33:29.736113 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.736116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.736161 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.736208 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.736808 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.737531 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.738552 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.738861 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.738935 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.741914 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:29.741920 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:29.741924 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:29.741947 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:29.743631 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.744113 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.744116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.744118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.744213 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.744560 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.744563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.744565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.744757 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:29.744829 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:29.745157 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 14:33:29.745282 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 14:33:29.746083 31742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43caba4d8b904590a3ecffb1905e1e9e: Neither blocks nor log segments found. Creating new log.
I20260808 14:33:29.746145 31828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.735358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.125.124:38677 (local address 127.0.0.1:51196)
0808 14:33:29.735899 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0808 14:33:29.735913 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:33:29.736020 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:33:29.736328 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:33:29.736332 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:33:29.736538 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:33:29.736686 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:29.736695 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.737657 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.737660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:29.738444 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:33:29.738454 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.739095 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.739098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:33:29.739171 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.740228 (+ 1057us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:33:29.740244 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:33:29.741761 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:33:29.743739 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:29.743745 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:29.743748 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:29.744004 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:29.744381 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:33:29.744384 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:33:29.744386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:33:29.744467 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:33:29.744895 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:33:29.744897 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:33:29.744990 (+ 93us) client_negotiation.cc:769] Sending connection context
0808 14:33:29.745582 (+ 592us) client_negotiation.cc:240] Negotiation successful
0808 14:33:29.745717 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468}
I20260808 14:33:29.747309 31658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d110290f8484d5693c4a1db44492a46" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36093 } } peers { permanent_uuid: "43caba4d8b904590a3ecffb1905e1e9e" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } } peers { permanent_uuid: "0a9924acf27646ab9dc0c32f4c66db3e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } }
I20260808 14:33:29.747610 31742 log.cc:824] T 00000000000000000000000000000000 P 43caba4d8b904590a3ecffb1905e1e9e: Log is configured to *not* fsync() on all Append() calls
I20260808 14:33:29.747676 31658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d110290f8484d5693c4a1db44492a46, State: Initialized, Role: FOLLOWER
I20260808 14:33:29.748102 31658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d110290f8484d5693c4a1db44492a46" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36093 } } peers { permanent_uuid: "43caba4d8b904590a3ecffb1905e1e9e" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } } peers { permanent_uuid: "0a9924acf27646ab9dc0c32f4c66db3e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } }
I20260808 14:33:29.748579 31832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d110290f8484d5693c4a1db44492a46" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36093 } } peers { permanent_uuid: "43caba4d8b904590a3ecffb1905e1e9e" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } } peers { permanent_uuid: "0a9924acf27646ab9dc0c32f4c66db3e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } } }
I20260808 14:33:29.748768 31832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:33:29.749342 31748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.736188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53480 (local address 127.7.125.125:41177)
0808 14:33:29.736291 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.736295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.736406 (+ 111us) server_negotiation.cc:407] Connection header received
0808 14:33:29.736648 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.736651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.736981 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.737036 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.738172 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.738889 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.740442 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.740789 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.740868 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.745492 (+ 4624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:29.745496 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:29.745498 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:29.745526 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:29.747261 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.747815 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.747819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.747821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.747903 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.748207 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.748210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.748212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.748417 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:29.748663 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:29.749085 (+ 422us) server_negotiation.cc:299] Negotiation successful
0808 14:33:29.749203 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 14:33:29.749425 31658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:33:29.749841 31824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a9924acf27646ab9dc0c32f4c66db3e: Bootstrap starting.
I20260808 14:33:29.752655 31824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a9924acf27646ab9dc0c32f4c66db3e: Neither blocks nor log segments found. Creating new log.
I20260808 14:33:29.753337 7669 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:33:29.753563 31824 log.cc:824] T 00000000000000000000000000000000 P 0a9924acf27646ab9dc0c32f4c66db3e: Log is configured to *not* fsync() on all Append() calls
I20260808 14:33:29.754587 31631 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5d110290f8484d5693c4a1db44492a46"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42744
W20260808 14:33:29.754714 31843 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:33:29.754817 31843 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:33:29.754822 31631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 14:33:29.754966 31631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:33:29.756292 31631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d110290f8484d5693c4a1db44492a46" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36093 } } peers { permanent_uuid: "43caba4d8b904590a3ecffb1905e1e9e" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } } peers { permanent_uuid: "0a9924acf27646ab9dc0c32f4c66db3e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } }
I20260808 14:33:29.757242 31631 leader_election.cc:288] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46 [CANDIDATE]: Term 1 election: Requested vote from peers 43caba4d8b904590a3ecffb1905e1e9e (127.7.125.125:41177), 0a9924acf27646ab9dc0c32f4c66db3e (127.7.125.124:38677)
I20260808 14:33:29.754329 31587 master_runner.cc:186] Error getting master registration for 127.7.125.125:41177: OK, instance_id { permanent_uuid: "43caba4d8b904590a3ecffb1905e1e9e" instance_seqno: 1786199609473717 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 14:33:29.757782 31587 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 14:33:29.760077 31824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a9924acf27646ab9dc0c32f4c66db3e: No bootstrap required, opened a new log
I20260808 14:33:29.760075 31742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43caba4d8b904590a3ecffb1905e1e9e: No bootstrap required, opened a new log
I20260808 14:33:29.764227 31742 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43caba4d8b904590a3ecffb1905e1e9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d110290f8484d5693c4a1db44492a46" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36093 } } peers { permanent_uuid: "43caba4d8b904590a3ecffb1905e1e9e" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } } peers { permanent_uuid: "0a9924acf27646ab9dc0c32f4c66db3e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } }
I20260808 14:33:29.764248 31824 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a9924acf27646ab9dc0c32f4c66db3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d110290f8484d5693c4a1db44492a46" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36093 } } peers { permanent_uuid: "43caba4d8b904590a3ecffb1905e1e9e" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } } peers { permanent_uuid: "0a9924acf27646ab9dc0c32f4c66db3e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } }
I20260808 14:33:29.764508 31742 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43caba4d8b904590a3ecffb1905e1e9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43caba4d8b904590a3ecffb1905e1e9e, State: Initialized, Role: FOLLOWER
I20260808 14:33:29.764570 31824 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a9924acf27646ab9dc0c32f4c66db3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a9924acf27646ab9dc0c32f4c66db3e, State: Initialized, Role: FOLLOWER
I20260808 14:33:29.764863 31742 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43caba4d8b904590a3ecffb1905e1e9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d110290f8484d5693c4a1db44492a46" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36093 } } peers { permanent_uuid: "43caba4d8b904590a3ecffb1905e1e9e" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } } peers { permanent_uuid: "0a9924acf27646ab9dc0c32f4c66db3e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } }
I20260808 14:33:29.764891 31824 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a9924acf27646ab9dc0c32f4c66db3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d110290f8484d5693c4a1db44492a46" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36093 } } peers { permanent_uuid: "43caba4d8b904590a3ecffb1905e1e9e" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } } peers { permanent_uuid: "0a9924acf27646ab9dc0c32f4c66db3e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } }
I20260808 14:33:29.765489 31850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a9924acf27646ab9dc0c32f4c66db3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d110290f8484d5693c4a1db44492a46" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36093 } } peers { permanent_uuid: "43caba4d8b904590a3ecffb1905e1e9e" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } } peers { permanent_uuid: "0a9924acf27646ab9dc0c32f4c66db3e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } } }
I20260808 14:33:29.765815 31850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a9924acf27646ab9dc0c32f4c66db3e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:33:29.766325 31824 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a9924acf27646ab9dc0c32f4c66db3e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:33:29.766327 31742 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43caba4d8b904590a3ecffb1905e1e9e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:33:29.766387 31795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5d110290f8484d5693c4a1db44492a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0a9924acf27646ab9dc0c32f4c66db3e"
I20260808 14:33:29.766584 31795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a9924acf27646ab9dc0c32f4c66db3e [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:33:29.767446 31711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5d110290f8484d5693c4a1db44492a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "43caba4d8b904590a3ecffb1905e1e9e"
I20260808 14:33:29.767709 31711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43caba4d8b904590a3ecffb1905e1e9e [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:33:29.769212 31711 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 43caba4d8b904590a3ecffb1905e1e9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d110290f8484d5693c4a1db44492a46 in term 1.
I20260808 14:33:29.770435 31604 leader_election.cc:302] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43caba4d8b904590a3ecffb1905e1e9e, 5d110290f8484d5693c4a1db44492a46; no voters:
I20260808 14:33:29.770784 31832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:33:29.771080 31832 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46 [term 1 LEADER]: Becoming Leader. State: Replica: 5d110290f8484d5693c4a1db44492a46, State: Running, Role: LEADER
I20260808 14:33:29.771458 31832 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d110290f8484d5693c4a1db44492a46" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36093 } } peers { permanent_uuid: "43caba4d8b904590a3ecffb1905e1e9e" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } } peers { permanent_uuid: "0a9924acf27646ab9dc0c32f4c66db3e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } }
I20260808 14:33:29.772535 31795 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0a9924acf27646ab9dc0c32f4c66db3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d110290f8484d5693c4a1db44492a46 in term 1.
I20260808 14:33:29.773569 31832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d110290f8484d5693c4a1db44492a46. Latest consensus state: current_term: 1 leader_uuid: "5d110290f8484d5693c4a1db44492a46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d110290f8484d5693c4a1db44492a46" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36093 } } peers { permanent_uuid: "43caba4d8b904590a3ecffb1905e1e9e" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } } peers { permanent_uuid: "0a9924acf27646ab9dc0c32f4c66db3e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } } }
I20260808 14:33:29.773710 31832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:29.780575 31851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43caba4d8b904590a3ecffb1905e1e9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d110290f8484d5693c4a1db44492a46" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36093 } } peers { permanent_uuid: "43caba4d8b904590a3ecffb1905e1e9e" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } } peers { permanent_uuid: "0a9924acf27646ab9dc0c32f4c66db3e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } } }
I20260808 14:33:29.780707 31851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43caba4d8b904590a3ecffb1905e1e9e [sys.catalog]: This master's current role is: FOLLOWER
W20260808 14:33:29.783382 31881 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a9924acf27646ab9dc0c32f4c66db3e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:33:29.783463 31881 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:33:29.785881 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.769563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42752 (local address 127.7.125.126:36093)
0808 14:33:29.769857 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.769864 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.769883 (+ 19us) server_negotiation.cc:407] Connection header received
0808 14:33:29.769930 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.769933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.769975 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.770050 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.774489 (+ 4439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.775226 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.776506 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.776814 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.776909 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.779942 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:29.779946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:29.779948 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:29.779976 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:29.781695 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.782236 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.782241 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.782243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.782340 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.782940 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.782943 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.782948 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.784840 (+ 1892us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:29.784987 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:29.785580 (+ 593us) server_negotiation.cc:299] Negotiation successful
0808 14:33:29.785703 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180}
I20260808 14:33:29.786430 31827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.768828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51198 (local address 127.7.125.124:38677)
0808 14:33:29.768971 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.768975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.775991 (+ 7016us) server_negotiation.cc:407] Connection header received
0808 14:33:29.776330 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.776333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.776489 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.776554 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.777814 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.778496 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.779570 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.779889 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.779988 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.782257 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:29.782261 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:29.782263 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:29.782289 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:29.784459 (+ 2170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.784982 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.784986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.784988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.785074 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.785380 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.785384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.785385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.785588 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:29.785683 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:29.786173 (+ 490us) server_negotiation.cc:299] Negotiation successful
0808 14:33:29.786295 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 14:33:29.786504 31748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:29.767438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53484 (local address 127.7.125.125:41177)
0808 14:33:29.767565 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.767570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.769324 (+ 1754us) server_negotiation.cc:407] Connection header received
0808 14:33:29.769485 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.769489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.769540 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.769591 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.770661 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.771476 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.772697 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:29.773005 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:29.773110 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:29.782722 (+ 9612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:33:29.782727 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:33:29.782730 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:33:29.782761 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:33:29.784651 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.785130 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.785134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.785136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.785232 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:33:29.785554 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:33:29.785557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:33:29.785559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:33:29.785773 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:33:29.785868 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:29.786237 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 14:33:29.786375 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 14:33:29.788009 31882 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:33:29.788928 31882 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260808 14:33:29.791666 31877 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 43caba4d8b904590a3ecffb1905e1e9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:33:29.793931 31877 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:33:29.794689 31711 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 43caba4d8b904590a3ecffb1905e1e9e [term 1 FOLLOWER]: Refusing update from remote peer 5d110290f8484d5693c4a1db44492a46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:33:29.795797 31795 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0a9924acf27646ab9dc0c32f4c66db3e [term 1 FOLLOWER]: Refusing update from remote peer 5d110290f8484d5693c4a1db44492a46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:33:29.797765 31832 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43caba4d8b904590a3ecffb1905e1e9e" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:33:29.798189 31832 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a9924acf27646ab9dc0c32f4c66db3e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:33:29.802160 31850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a9924acf27646ab9dc0c32f4c66db3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d110290f8484d5693c4a1db44492a46. Latest consensus state: current_term: 1 leader_uuid: "5d110290f8484d5693c4a1db44492a46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d110290f8484d5693c4a1db44492a46" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36093 } } peers { permanent_uuid: "43caba4d8b904590a3ecffb1905e1e9e" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } } peers { permanent_uuid: "0a9924acf27646ab9dc0c32f4c66db3e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } } }
I20260808 14:33:29.802306 31850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a9924acf27646ab9dc0c32f4c66db3e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:33:29.808327 31851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43caba4d8b904590a3ecffb1905e1e9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d110290f8484d5693c4a1db44492a46. Latest consensus state: current_term: 1 leader_uuid: "5d110290f8484d5693c4a1db44492a46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d110290f8484d5693c4a1db44492a46" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36093 } } peers { permanent_uuid: "43caba4d8b904590a3ecffb1905e1e9e" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } } peers { permanent_uuid: "0a9924acf27646ab9dc0c32f4c66db3e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } } }
I20260808 14:33:29.808499 31851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43caba4d8b904590a3ecffb1905e1e9e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:33:29.818367 31887 mvcc.cc:205] Tried to move back new op lower bound from 7316273601711075328 to 7316273601627865088. Current Snapshot: MvccSnapshot[applied={T|T < 7316273601711075328}]
I20260808 14:33:29.819705 31850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a9924acf27646ab9dc0c32f4c66db3e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5d110290f8484d5693c4a1db44492a46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d110290f8484d5693c4a1db44492a46" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36093 } } peers { permanent_uuid: "43caba4d8b904590a3ecffb1905e1e9e" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } } peers { permanent_uuid: "0a9924acf27646ab9dc0c32f4c66db3e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } } }
I20260808 14:33:29.819844 31850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a9924acf27646ab9dc0c32f4c66db3e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:33:29.820426 31832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5d110290f8484d5693c4a1db44492a46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d110290f8484d5693c4a1db44492a46" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36093 } } peers { permanent_uuid: "43caba4d8b904590a3ecffb1905e1e9e" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } } peers { permanent_uuid: "0a9924acf27646ab9dc0c32f4c66db3e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } } }
I20260808 14:33:29.820549 31832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:29.822575 31851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43caba4d8b904590a3ecffb1905e1e9e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5d110290f8484d5693c4a1db44492a46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d110290f8484d5693c4a1db44492a46" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36093 } } peers { permanent_uuid: "43caba4d8b904590a3ecffb1905e1e9e" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } } peers { permanent_uuid: "0a9924acf27646ab9dc0c32f4c66db3e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } } }
I20260808 14:33:29.822727 31851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43caba4d8b904590a3ecffb1905e1e9e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:33:29.823001 31832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5d110290f8484d5693c4a1db44492a46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d110290f8484d5693c4a1db44492a46" member_type: VOTER last_known_addr { host: "127.7.125.126" port: 36093 } } peers { permanent_uuid: "43caba4d8b904590a3ecffb1905e1e9e" member_type: VOTER last_known_addr { host: "127.7.125.125" port: 41177 } } peers { permanent_uuid: "0a9924acf27646ab9dc0c32f4c66db3e" member_type: VOTER last_known_addr { host: "127.7.125.124" port: 38677 } } }
I20260808 14:33:29.823117 31832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46 [sys.catalog]: This master's current role is: LEADER
I20260808 14:33:29.823558 31888 mvcc.cc:205] Tried to move back new op lower bound from 7316273601711075328 to 7316273601627865088. Current Snapshot: MvccSnapshot[applied={T|T < 7316273601711075328}]
I20260808 14:33:29.823742 31890 mvcc.cc:205] Tried to move back new op lower bound from 7316273601711075328 to 7316273601627865088. Current Snapshot: MvccSnapshot[applied={T|T < 7316273601711075328}]
I20260808 14:33:29.827910 31882 catalog_manager.cc:1347] Generated new cluster ID: d90a5a101ab14e838ca7065a4d6936a7
I20260808 14:33:29.827987 31882 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:33:29.858090 31882 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:33:29.859591 31882 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:33:29.876976 31882 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d110290f8484d5693c4a1db44492a46: Generated new TSK 0
I20260808 14:33:29.877933 31882 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 14:33:29.910944 31661 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:29.908260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42754 (local address 127.7.125.126:36093)
0808 14:33:29.908409 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.908414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.909748 (+ 1334us) server_negotiation.cc:407] Connection header received
0808 14:33:29.909991 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.909995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.910051 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.910107 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.910812 (+ 705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42754: BlockingRecv error: recv got EOF from 127.0.0.1:42754 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 14:33:29.912428 31827 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:29.909083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51206 (local address 127.7.125.124:38677)
0808 14:33:29.909209 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.909214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.911498 (+ 2284us) server_negotiation.cc:407] Connection header received
0808 14:33:29.911684 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.911687 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.911738 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.911790 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.912294 (+ 504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51206: BlockingRecv error: recv got EOF from 127.0.0.1:51206 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 14:33:29.914489 31748 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:29.912898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53486 (local address 127.7.125.125:41177)
0808 14:33:29.913015 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.913019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.913549 (+ 530us) server_negotiation.cc:407] Connection header received
0808 14:33:29.913732 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.913736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.913790 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.913845 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.914350 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53486: BlockingRecv error: recv got EOF from 127.0.0.1:53486 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 14:33:29.919399 31748 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:29.917608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53492 (local address 127.7.125.125:41177)
0808 14:33:29.917714 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.917718 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.918240 (+ 522us) server_negotiation.cc:407] Connection header received
0808 14:33:29.918488 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.918492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.918572 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.918623 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.919280 (+ 657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53492: BlockingRecv error: recv got EOF from 127.0.0.1:53492 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 14:33:29.924747 31748 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:29.923132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53504 (local address 127.7.125.125:41177)
0808 14:33:29.923242 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.923247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.923744 (+ 497us) server_negotiation.cc:407] Connection header received
0808 14:33:29.923966 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.923971 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.924038 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.924105 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.924643 (+ 538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53504: BlockingRecv error: recv got EOF from 127.0.0.1:53504 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 14:33:29.930647 31748 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:29.929037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53512 (local address 127.7.125.125:41177)
0808 14:33:29.929147 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.929151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.929662 (+ 511us) server_negotiation.cc:407] Connection header received
0808 14:33:29.929876 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.929880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.929934 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.929989 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.930540 (+ 551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53512: BlockingRecv error: recv got EOF from 127.0.0.1:53512 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 14:33:29.938832 31748 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:29.937132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53516 (local address 127.7.125.125:41177)
0808 14:33:29.937276 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.937280 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.937811 (+ 531us) server_negotiation.cc:407] Connection header received
0808 14:33:29.938016 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.938022 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.938079 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.938137 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.938723 (+ 586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53516: BlockingRecv error: recv got EOF from 127.0.0.1:53516 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260808 14:33:29.952209 31748 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:29.949832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53524 (local address 127.7.125.125:41177)
0808 14:33:29.950002 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.950007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.950902 (+ 895us) server_negotiation.cc:407] Connection header received
0808 14:33:29.951262 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.951266 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.951333 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.951414 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.952078 (+ 664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53524: BlockingRecv error: recv got EOF from 127.0.0.1:53524 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 14:33:29.973855 31748 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:29.971937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53540 (local address 127.7.125.125:41177)
0808 14:33:29.972060 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:29.972065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:29.972752 (+ 687us) server_negotiation.cc:407] Connection header received
0808 14:33:29.972978 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:29.972982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:29.973043 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:29.973100 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:29.973730 (+ 630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53540: BlockingRecv error: recv got EOF from 127.0.0.1:53540 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 14:33:30.009979 31748 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:30.008616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53554 (local address 127.7.125.125:41177)
0808 14:33:30.008762 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:30.008767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:30.009019 (+ 252us) server_negotiation.cc:407] Connection header received
0808 14:33:30.009193 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:30.009196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:30.009255 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:30.009346 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:30.009871 (+ 525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53554: BlockingRecv error: recv got EOF from 127.0.0.1:53554 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 14:33:30.078229 31748 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:30.076780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53568 (local address 127.7.125.125:41177)
0808 14:33:30.076943 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:30.076949 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:30.077221 (+ 272us) server_negotiation.cc:407] Connection header received
0808 14:33:30.077427 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:30.077430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:30.077481 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:30.077579 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:30.078082 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53568: BlockingRecv error: recv got EOF from 127.0.0.1:53568 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 14:33:30.209941 31748 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:30.208740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53580 (local address 127.7.125.125:41177)
0808 14:33:30.208931 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:30.208935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:30.209042 (+ 107us) server_negotiation.cc:407] Connection header received
0808 14:33:30.209167 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:30.209169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:30.209217 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:30.209290 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:30.209766 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53580: BlockingRecv error: recv got EOF from 127.0.0.1:53580 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260808 14:33:30.469580 31748 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:33:30.468320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53588 (local address 127.7.125.125:41177)
0808 14:33:30.468479 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:30.468483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:30.468617 (+ 134us) server_negotiation.cc:407] Connection header received
0808 14:33:30.468805 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:30.468809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:30.468868 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:30.468965 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:33:30.469454 (+ 489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53588: BlockingRecv error: recv got EOF from 127.0.0.1:53588 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 14:33:30.784843 31881 catalog_manager.cc:1259] Loaded cluster ID: d90a5a101ab14e838ca7065a4d6936a7
I20260808 14:33:30.784940 31881 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0a9924acf27646ab9dc0c32f4c66db3e: loading cluster ID for follower catalog manager: success
I20260808 14:33:30.786664 31881 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0a9924acf27646ab9dc0c32f4c66db3e: acquiring CA information for follower catalog manager: success
I20260808 14:33:30.787916 31881 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0a9924acf27646ab9dc0c32f4c66db3e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 14:33:30.795377 31877 catalog_manager.cc:1259] Loaded cluster ID: d90a5a101ab14e838ca7065a4d6936a7
I20260808 14:33:30.795504 31877 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 43caba4d8b904590a3ecffb1905e1e9e: loading cluster ID for follower catalog manager: success
I20260808 14:33:30.797175 31877 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 43caba4d8b904590a3ecffb1905e1e9e: acquiring CA information for follower catalog manager: success
I20260808 14:33:30.798395 31877 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 43caba4d8b904590a3ecffb1905e1e9e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 14:33:30.988128 31966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:30.983881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53596 (local address 127.7.125.125:41177)
0808 14:33:30.984145 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:30.984149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:30.984268 (+ 119us) server_negotiation.cc:407] Connection header received
0808 14:33:30.984494 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:30.984498 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:30.984642 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:30.984767 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:33:30.985056 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:30.985588 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:30.986493 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:30.986731 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:30.986864 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:30.987244 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:33:30.987693 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:33:30.987840 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:30.987902 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 14:33:30.987963 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":71,"threads_started":1}
I20260808 14:33:30.994931 31974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:30.991432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51220 (local address 127.7.125.124:38677)
0808 14:33:30.991707 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:30.991711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:30.991723 (+ 12us) server_negotiation.cc:407] Connection header received
0808 14:33:30.991809 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:30.991811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:30.991926 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:30.992023 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:33:30.992193 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:30.992703 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:30.993514 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:30.993767 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:30.993865 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:30.994163 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:33:30.994568 (+ 405us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:33:30.994660 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:30.994746 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 14:33:30.994795 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":129,"threads_started":1}
I20260808 14:33:31.001847 31966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:30.998070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53604 (local address 127.7.125.125:41177)
0808 14:33:30.998215 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:30.998220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:30.998661 (+ 441us) server_negotiation.cc:407] Connection header received
0808 14:33:30.998782 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:30.998785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:30.998839 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:30.998913 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:33:30.999199 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:30.999780 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:31.000798 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:31.001026 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:31.001135 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:31.001379 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:33:31.001447 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:33:31.001527 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:31.001658 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 14:33:31.001713 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 14:33:31.002981 31974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:30.998446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51222 (local address 127.7.125.124:38677)
0808 14:33:30.998575 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:30.998578 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:30.998702 (+ 124us) server_negotiation.cc:407] Connection header received
0808 14:33:30.998851 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:30.998854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:30.998913 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:30.998996 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:33:30.999395 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:31.000175 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:31.001639 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:31.001869 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:31.001992 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:31.002488 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:33:31.002570 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:33:31.002620 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:31.002772 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 14:33:31.002825 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 14:33:31.003464 31982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:33:30.998028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42760 (local address 127.7.125.126:36093)
0808 14:33:30.998397 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0808 14:33:30.998402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:33:30.998418 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:33:30.998627 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:33:30.998632 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:33:30.998798 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:33:30.998954 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:33:30.999308 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:31.000200 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:31.001602 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:33:31.001939 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:33:31.002040 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:33:31.002452 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:33:31.002790 (+ 338us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:33:31.002884 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 14:33:31.003253 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 14:33:31.003315 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":81,"threads_started":1}
I20260808 14:33:31.006516 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 31587
I20260808 14:33:31.015331 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 31665
I20260808 14:33:31.022112 7669 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskngm_Z6/build/debug/bin/kudu with pid 31750
2026-08-08T14:33:31Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3473 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6679 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (294121 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST